Task 15489238

Name	hadcm3n_3ipf_1940_40_008261116_0
Workunit	8416240
Created	20 Dec 2012, 20:47:47 UTC
Sent	20 Dec 2012, 20:52:24 UTC
Report deadline	22 Mar 2013, 4:19:35 UTC
Received	14 Jan 2013, 20:29:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1027037
Run time	9 days 4 hours 37 min 24 sec
CPU time	7 days 13 hours 15 min 55 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.75 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 11:47:02 (6524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:47:03 (6524): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:47:23 (6264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:13:04 (2848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7136, iMonCtr=1 Model crash detected, will try to restart... 08:24:48 (4652): No heartbeat from core client for 30 sec - exiting 08:24:49 (4652): No heartbeat from core client for 30 sec - exiting 08:24:50 (4652): No heartbeat from core client for 30 sec - exiting 08:24:51 (4652): No heartbeat from core client for 30 sec - exiting 08:24:52 (4652): No heartbeat from core client for 30 sec - exiting 08:24:53 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:18:47 (5628): No heartbeat from core client for 30 sec - exiting 08:18:49 (5628): No heartbeat from core client for 30 sec - exiting 08:18:50 (5628): No heartbeat from core client for 30 sec - exiting 08:18:51 (5628): No heartbeat from core client for 30 sec - exiting 08:18:52 (5628): No heartbeat from core client for 30 sec - exiting 08:18:53 (5628): No heartbeat from core client for 30 sec - exiting 08:18:54 (5628): No heartbeat from core client for 30 sec - exiting 08:18:55 (5628): No heartbeat from core client for 30 sec - exiting 08:18:56 (5628): No heartbeat from core client for 30 sec - exiting 08:18:57 (5628): No heartbeat from core client for 30 sec - exiting 08:18:58 (5628): No heartbeat from core client for 30 sec - exiting 08:18:59 (5628): No heartbeat from core client for 30 sec - exiting 08:19:00 (5628): No heartbeat from core client for 30 sec - exiting 08:19:01 (5628): No heartbeat from core client for 30 sec - exiting 08:19:02 (5628): No heartbeat from core client for 30 sec - exiting 08:19:03 (5628): No heartbeat from core client for 30 sec - exiting 08:19:04 (5628): No heartbeat from core client for 30 sec - exiting 08:19:05 (5628): No heartbeat from core client for 30 sec - exiting 08:19:06 (5628): No heartbeat from core client for 30 sec - exiting 08:19:07 (5628): No heartbeat from core client for 30 sec - exiting 08:19:08 (5628): No heartbeat from core client for 30 sec - exiting 08:19:09 (5628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:40:52 (6140): No heartbeat from core client for 30 sec - exiting 08:40:53 (6140): No heartbeat from core client for 30 sec - exiting 08:40:54 (6140): No heartbeat from core client for 30 sec - exiting 08:40:55 (6140): No heartbeat from core client for 30 sec - exiting 08:40:56 (6140): No heartbeat from core client for 30 sec - exiting 08:40:57 (6140): No heartbeat from core client for 30 sec - exiting 08:40:58 (6140): No heartbeat from core client for 30 sec - exiting 08:40:59 (6140): No heartbeat from core client for 30 sec - exiting 08:41:00 (6140): No heartbeat from core client for 30 sec - exiting 08:41:01 (6140): No heartbeat from core client for 30 sec - exiting 08:41:02 (6140): No heartbeat from core client for 30 sec - exiting 08:41:03 (6140): No heartbeat from core client for 30 sec - exiting 08:41:04 (6140): No heartbeat from core client for 30 sec - exiting 08:41:05 (6140): No heartbeat from core client for 30 sec - exiting 08:41:06 (6140): No heartbeat from core client for 30 sec - exiting 08:41:07 (6140): No heartbeat from core client for 30 sec - exiting 08:41:08 (6140): No heartbeat from core client for 30 sec - exiting 08:41:09 (6140): No heartbeat from core client for 30 sec - exiting 08:41:10 (6140): No heartbeat from core client for 30 sec - exiting 08:41:11 (6140): No heartbeat from core client for 30 sec - exiting 08:41:12 (6140): No heartbeat from core client for 30 sec - exiting 08:41:13 (6140): No heartbeat from core client for 30 sec - exiting 08:41:14 (6140): No heartbeat from core client for 30 sec - exiting 08:41:15 (6140): No heartbeat from core client for 30 sec - exiting 08:41:16 (6140): No heartbeat from core client for 30 sec - exiting 08:41:17 (6140): No heartbeat from core client for 30 sec - exiting 08:41:18 (6140): No heartbeat from core client for 30 sec - exiting 08:41:19 (6140): No heartbeat from core client for 30 sec - exiting 08:41:20 (6140): No heartbeat from core client for 30 sec - exiting 08:41:21 (6140): No heartbeat from core client for 30 sec - exiting 08:41:22 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:18:08 (5896): No heartbeat from core client for 30 sec - exiting 08:18:09 (5896): No heartbeat from core client for 30 sec - exiting 08:18:10 (5896): No heartbeat from core client for 30 sec - exiting 08:18:11 (5896): No heartbeat from core client for 30 sec - exiting 08:18:12 (5896): No heartbeat from core client for 30 sec - exiting 08:18:13 (5896): No heartbeat from core client for 30 sec - exiting 08:18:14 (5896): No heartbeat from core client for 30 sec - exiting 08:18:15 (5896): No heartbeat from core client for 30 sec - exiting 08:18:16 (5896): No heartbeat from core client for 30 sec - exiting 08:18:17 (5896): No heartbeat from core client for 30 sec - exiting 08:18:18 (5896): No heartbeat from core client for 30 sec - exiting 08:18:19 (5896): No heartbeat from core client for 30 sec - exiting 08:18:20 (5896): No heartbeat from core client for 30 sec - exiting 08:18:21 (5896): No heartbeat from core client for 30 sec - exiting 08:18:22 (5896): No heartbeat from core client for 30 sec - exiting 08:18:23 (5896): No heartbeat from core client for 30 sec - exiting 08:18:24 (5896): No heartbeat from core client for 30 sec - exiting 08:18:25 (5896): No heartbeat from core client for 30 sec - exiting 08:18:26 (5896): No heartbeat from core client for 30 sec - exiting 08:18:27 (5896): No heartbeat from core client for 30 sec - exiting 08:18:28 (5896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:18:29 (5896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:35:24 (5168): No heartbeat from core client for 30 sec - exiting 19:35:25 (5168): No heartbeat from core client for 30 sec - exiting 19:35:26 (5168): No heartbeat from core client for 30 sec - exiting 19:35:27 (5168): No heartbeat from core client for 30 sec - exiting 19:35:28 (5168): No heartbeat from core client for 30 sec - exiting 19:35:29 (5168): No heartbeat from core client for 30 sec - exiting 19:35:30 (5168): No heartbeat from core client for 30 sec - exiting 19:35:31 (5168): No heartbeat from core client for 30 sec - exiting 19:35:32 (5168): No heartbeat from core client for 30 sec - exiting 19:35:33 (5168): No heartbeat from core client for 30 sec - exiting 19:35:34 (5168): No heartbeat from core client for 30 sec - exiting 19:35:35 (5168): No heartbeat from core client for 30 sec - exiting 19:35:36 (5168): No heartbeat from core client for 30 sec - exiting 19:35:37 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:35:38 (5168): No heartbeat from core client for 30 sec - exiting 19:35:39 (5168): No heartbeat from core client for 30 sec - exiting 19:35:40 (5168): No heartbeat from core client for 30 sec - exiting 19:35:41 (5168): No heartbeat from core client for 30 sec - exiting 19:35:42 (5168): No heartbeat from core client for 30 sec - exiting 19:35:43 (5168): No heartbeat from core client for 30 sec - exiting 19:35:44 (5168): No heartbeat from core client for 30 sec - exiting 19:35:45 (5168): No heartbeat from core client for 30 sec - exiting 19:35:46 (5168): No heartbeat from core client for 30 sec - exiting 19:35:47 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 07:31:56 (6976): No heartbeat from core client for 30 sec - exiting 07:31:57 (6976): No heartbeat from core client for 30 sec - exiting 07:31:58 (6976): No heartbeat from core client for 30 sec - exiting 07:31:59 (6976): No heartbeat from core client for 30 sec - exiting 07:32:00 (6976): No heartbeat from core client for 30 sec - exiting 07:32:01 (6976): No heartbeat from core client for 30 sec - exiting 07:32:02 (6976): No heartbeat from core client for 30 sec - exiting 07:32:03 (6976): No heartbeat from core client for 30 sec - exiting 07:32:04 (6976): No heartbeat from core client for 30 sec - exiting 07:32:05 (6976): No heartbeat from core client for 30 sec - exiting 07:32:06 (6976): No heartbeat from core client for 30 sec - exiting 07:32:07 (6976): No heartbeat from core client for 30 sec - exiting 07:32:08 (6976): No heartbeat from core client for 30 sec - exiting 07:32:09 (6976): No heartbeat from core client for 30 sec - exiting 07:32:10 (6976): No heartbeat from core client for 30 sec - exiting 07:32:11 (6976): No heartbeat from core client for 30 sec - exiting 07:32:12 (6976): No heartbeat from core client for 30 sec - exiting 07:32:13 (6976): No heartbeat from core client for 30 sec - exiting 07:32:14 (6976): No heartbeat from core client for 30 sec - exiting 07:32:15 (6976): No heartbeat from core client for 30 sec - exiting 07:32:16 (6976): No heartbeat from core client for 30 sec - exiting 07:32:17 (6976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 09:15:40 (5576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:15:41 (5576): No heartbeat from core client for 30 sec - exiting 09:54:26 (4068): No heartbeat from core client for 30 sec - exiting 09:54:27 (4068): No heartbeat from core client for 30 sec - exiting 09:54:28 (4068): No heartbeat from core client for 30 sec - exiting 09:54:29 (4068): No heartbeat from core client for 30 sec - exiting 09:54:30 (4068): No heartbeat from core client for 30 sec - exiting 09:54:31 (4068): No heartbeat from core client for 30 sec - exiting 09:54:32 (4068): No heartbeat from core client for 30 sec - exiting 09:54:33 (4068): No heartbeat from core client for 30 sec - exiting 09:54:34 (4068): No heartbeat from core client for 30 sec - exiting 09:54:35 (4068): No heartbeat from core client for 30 sec - exiting 09:54:36 (4068): No heartbeat from core client for 30 sec - exiting 09:54:37 (4068): No heartbeat from core client for 30 sec - exiting 09:54:38 (4068): No heartbeat from core client for 30 sec - exiting 09:54:39 (4068): No heartbeat from core client for 30 sec - exiting 09:54:40 (4068): No heartbeat from core client for 30 sec - exiting 09:54:41 (4068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:04:22 (7972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:04:23 (7972): No heartbeat from core client for 30 sec - exiting 10:31:11 (3532): No heartbeat from core client for 30 sec - exiting 10:31:12 (3532): No heartbeat from core client for 30 sec - exiting 10:31:13 (3532): No heartbeat from core client for 30 sec - exiting 10:31:14 (3532): No heartbeat from core client for 30 sec - exiting 10:31:15 (3532): No heartbeat from core client for 30 sec - exiting 10:31:16 (3532): No heartbeat from core client for 30 sec - exiting 10:31:17 (3532): No heartbeat from core client for 30 sec - exiting 10:31:18 (3532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:51:28 (6372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:41:15 (5048): No heartbeat from core client for 30 sec - exiting 08:41:16 (5048): No heartbeat from core client for 30 sec - exiting 08:41:17 (5048): No heartbeat from core client for 30 sec - exiting 08:41:18 (5048): No heartbeat from core client for 30 sec - exiting 08:41:19 (5048): No heartbeat from core client for 30 sec - exiting 08:41:20 (5048): No heartbeat from core client for 30 sec - exiting 08:41:21 (5048): No heartbeat from core client for 30 sec - exiting 08:41:22 (5048): No heartbeat from core client for 30 sec - exiting 08:41:23 (5048): No heartbeat from core client for 30 sec - exiting 08:41:24 (5048): No heartbeat from core client for 30 sec - exiting 08:41:25 (5048): No heartbeat from core client for 30 sec - exiting 08:41:26 (5048): No heartbeat from core client for 30 sec - exiting 08:41:27 (5048): No heartbeat from core client for 30 sec - exiting 08:41:28 (5048): No heartbeat from core client for 30 sec - exiting 08:41:29 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:14:15 (6664): No heartbeat from core client for 30 sec - exiting 08:14:16 (6664): No heartbeat from core client for 30 sec - exiting 08:14:17 (6664): No heartbeat from core client for 30 sec - exiting 08:14:18 (6664): No heartbeat from core client for 30 sec - exiting 08:14:19 (6664): No heartbeat from core client for 30 sec - exiting 08:14:20 (6664): No heartbeat from core client for 30 sec - exiting 08:14:21 (6664): No heartbeat from core client for 30 sec - exiting 08:14:22 (6664): No heartbeat from core client for 30 sec - exiting 08:14:23 (6664): No heartbeat from core client for 30 sec - exiting 08:14:24 (6664): No heartbeat from core client for 30 sec - exiting 08:14:25 (6664): No heartbeat from core client for 30 sec - exiting 08:14:26 (6664): No heartbeat from core client for 30 sec - exiting 08:14:27 (6664): No heartbeat from core client for 30 sec - exiting 08:14:28 (6664): No heartbeat from core client for 30 sec - exiting 08:14:29 (6664): No heartbeat from core client for 30 sec - exiting 08:14:30 (6664): No heartbeat from core client for 30 sec - exiting 08:14:31 (6664): No heartbeat from core client for 30 sec - exiting 08:14:32 (6664): No heartbeat from core client for 30 sec - exiting 08:14:33 (6664): No heartbeat from core client for 30 sec - exiting 08:14:34 (6664): No heartbeat from core client for 30 sec - exiting 08:14:35 (6664): No heartbeat from core client for 30 sec - exiting 08:14:36 (6664): No heartbeat from core client for 30 sec - exiting 08:14:37 (6664): No heartbeat from core client for 30 sec - exiting 08:14:38 (6664): No heartbeat from core client for 30 sec - exiting 08:14:39 (6664): No heartbeat from core client for 30 sec - exiting 08:14:40 (6664): No heartbeat from core client for 30 sec - exiting 08:14:41 (6664): No heartbeat from core client for 30 sec - exiting 08:14:42 (6664): No heartbeat from core client for 30 sec - exiting 08:14:43 (6664): No heartbeat from core client for 30 sec - exiting 08:14:44 (6664): No heartbeat from core client for 30 sec - exiting 08:14:45 (6664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:53:11 (1976): No heartbeat from core client for 30 sec - exiting 08:53:12 (1976): No heartbeat from core client for 30 sec - exiting 08:53:13 (1976): No heartbeat from core client for 30 sec - exiting 08:53:14 (1976): No heartbeat from core client for 30 sec - exiting 08:53:15 (1976): No heartbeat from core client for 30 sec - exiting 08:53:16 (1976): No heartbeat from core client for 30 sec - exiting 08:53:17 (1976): No heartbeat from core client for 30 sec - exiting 08:53:18 (1976): No heartbeat from core client for 30 sec - exiting 08:53:19 (1976): No heartbeat from core client for 30 sec - exiting 08:53:20 (1976): No heartbeat from core client for 30 sec - exiting 08:53:21 (1976): No heartbeat from core client for 30 sec - exiting 08:53:22 (1976): No heartbeat from core client for 30 sec - exiting 08:53:23 (1976): No heartbeat from core client for 30 sec - exiting 08:53:24 (1976): No heartbeat from core client for 30 sec - exiting 08:53:25 (1976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:26 (1976): No heartbeat from core client for 30 sec - exiting 08:54:26 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6644, iMonCtr=1 Model crash detected, will try to restart... 17:49:02 (3392): No heartbeat from core client for 30 sec - exiting 17:49:03 (3392): No heartbeat from core client for 30 sec - exiting 17:49:04 (3392): No heartbeat from core client for 30 sec - exiting 17:49:05 (3392): No heartbeat from core client for 30 sec - exiting 17:49:06 (3392): No heartbeat from core client for 30 sec - exiting 17:49:07 (3392): No heartbeat from core client for 30 sec - exiting 17:49:08 (3392): No heartbeat from core client for 30 sec - exiting 17:49:09 (3392): No heartbeat from core client for 30 sec - exiting 17:49:10 (3392): No heartbeat from core client for 30 sec - exiting 17:49:11 (3392): No heartbeat from core client for 30 sec - exiting 17:49:12 (3392): No heartbeat from core client for 30 sec - exiting 17:49:13 (3392): No heartbeat from core client for 30 sec - exiting 17:49:14 (3392): No heartbeat from core client for 30 sec - exiting 17:49:15 (3392): No heartbeat from core client for 30 sec - exiting 17:49:16 (3392): No heartbeat from core client for 30 sec - exiting 17:49:17 (3392): No heartbeat from core client for 30 sec - exiting 17:49:18 (3392): No heartbeat from core client for 30 sec - exiting 17:49:19 (3392): No heartbeat from core client for 30 sec - exiting 17:49:20 (3392): No heartbeat from core client for 30 sec - exiting 17:49:21 (3392): No heartbeat from core client for 30 sec - exiting 17:49:22 (3392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:01:32 (6112): No heartbeat from core client for 30 sec - exiting 08:01:33 (6112): No heartbeat from core client for 30 sec - exiting 08:01:34 (6112): No heartbeat from core client for 30 sec - exiting 08:01:35 (6112): No heartbeat from core client for 30 sec - exiting 08:01:36 (6112): No heartbeat from core client for 30 sec - exiting 08:01:38 (6112): No heartbeat from core client for 30 sec - exiting 08:01:39 (6112): No heartbeat from core client for 30 sec - exiting 08:01:40 (6112): No heartbeat from core client for 30 sec - exiting 08:01:41 (6112): No heartbeat from core client for 30 sec - exiting 08:01:42 (6112): No heartbeat from core client for 30 sec - exiting 08:01:43 (6112): No heartbeat from core client for 30 sec - exiting 08:01:44 (6112): No heartbeat from core client for 30 sec - exiting 08:01:45 (6112): No heartbeat from core client for 30 sec - exiting 08:01:46 (6112): No heartbeat from core client for 30 sec - exiting 08:01:47 (6112): No heartbeat from core client for 30 sec - exiting 08:01:48 (6112): No heartbeat from core client for 30 sec - exiting 08:01:49 (6112): No heartbeat from core client for 30 sec - exiting 08:01:50 (6112): No heartbeat from core client for 30 sec - exiting 08:01:51 (6112): No heartbeat from core client for 30 sec - exiting 08:01:52 (6112): No heartbeat from core client for 30 sec - exiting 08:01:53 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:01:54 (6112): No heartbeat from core client for 30 sec - exiting 08:01:55 (6112): No heartbeat from core client for 30 sec - exiting 08:01:56 (6112): No heartbeat from core client for 30 sec - exiting 08:01:57 (6112): No heartbeat from core client for 30 sec - exiting 08:01:58 (6112): No heartbeat from core client for 30 sec - exiting 08:01:59 (6112): No heartbeat from core client for 30 sec - exiting 08:02:00 (6112): No heartbeat from core client for 30 sec - exiting 08:02:01 (6112): No heartbeat from core client for 30 sec - exiting 08:02:02 (6112): No heartbeat from core client for 30 sec - exiting 08:02:04 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Jan 2013 20:33:26	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	259,200	652,535	2.5175
10 Jan 2013 02:12:32	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	233,280	587,921	2.5202
06 Jan 2013 16:57:54	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	207,360	523,008	2.5222
05 Jan 2013 02:34:25	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	181,440	459,350	2.5317
03 Jan 2013 19:34:58	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	155,520	393,443	2.5299
02 Jan 2013 02:53:52	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	129,600	329,521	2.5426
27 Dec 2012 02:28:03	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	103,680	266,610	2.5715
25 Dec 2012 19:44:16	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	77,760	199,138	2.5609
24 Dec 2012 01:59:44	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	51,840	131,040	2.5278
22 Dec 2012 15:20:45	1027037	15489238	hadcm3n_3ipf_1940_40_008261116_0	25,920	66,635	2.5708