Task 13144543

Name	hadcm3n_yc9b_1900_40_007348745_2
Workunit	7546175
Created	18 Jul 2011, 4:34:55 UTC
Sent	18 Jul 2011, 4:46:28 UTC
Report deadline	17 Oct 2011, 12:13:39 UTC
Received	27 Feb 2012, 16:20:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1090407
Run time	61 days 23 hours 8 min 51 sec
CPU time	39 days 4 hours 36 min 50 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:34:43 (1228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:44 (1228): No heartbeat from core client for 30 sec - exiting 16:34:45 (1228): No heartbeat from core client for 30 sec - exiting 16:34:46 (1228): No heartbeat from core client for 30 sec - exiting 16:34:47 (1228): No heartbeat from core client for 30 sec - exiting 16:34:48 (1228): No heartbeat from core client for 30 sec - exiting 16:34:49 (1228): No heartbeat from core client for 30 sec - exiting 16:34:50 (1228): No heartbeat from core client for 30 sec - exiting 16:34:51 (1228): No heartbeat from core client for 30 sec - exiting 16:34:52 (1228): No heartbeat from core client for 30 sec - exiting 16:34:53 (1228): No heartbeat from core client for 30 sec - exiting 16:34:54 (1228): No heartbeat from core client for 30 sec - exiting 16:34:55 (1228): No heartbeat from core client for 30 sec - exiting 16:34:56 (1228): No heartbeat from core client for 30 sec - exiting 16:34:57 (1228): No heartbeat from core client for 30 sec - exiting 16:34:58 (1228): No heartbeat from core client for 30 sec - exiting 16:35:00 (1228): No heartbeat from core client for 30 sec - exiting 16:35:01 (1228): No heartbeat from core client for 30 sec - exiting 16:35:02 (1228): No heartbeat from core client for 30 sec - exiting 16:35:03 (1228): No heartbeat from core client for 30 sec - exiting 16:35:04 (1228): No heartbeat from core client for 30 sec - exiting 16:35:05 (1228): No heartbeat from core client for 30 sec - exiting 16:35:06 (1228): No heartbeat from core client for 30 sec - exiting 16:35:07 (1228): No heartbeat from core client for 30 sec - exiting 16:35:08 (1228): No heartbeat from core client for 30 sec - exiting 16:35:09 (1228): No heartbeat from core client for 30 sec - exiting 16:35:10 (1228): No heartbeat from core client for 30 sec - exiting 16:35:11 (1228): No heartbeat from core client for 30 sec - exiting 16:35:13 (1228): No heartbeat from core client for 30 sec - exiting 16:35:14 (1228): No heartbeat from core client for 30 sec - exiting 16:35:15 (1228): No heartbeat from core client for 30 sec - exiting 16:35:16 (1228): No heartbeat from core client for 30 sec - exiting 16:35:17 (1228): No heartbeat from core client for 30 sec - exiting 16:35:18 (1228): No heartbeat from core client for 30 sec - exiting 16:35:19 (1228): No heartbeat from core client for 30 sec - exiting 16:35:20 (1228): No heartbeat from core client for 30 sec - exiting 16:35:21 (1228): No heartbeat from core client for 30 sec - exiting 16:35:22 (1228): No heartbeat from core client for 30 sec - exiting 16:35:24 (1228): No heartbeat from core client for 30 sec - exiting 16:35:25 (1228): No heartbeat from core client for 30 sec - exiting 16:35:26 (1228): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:47:51 (10964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:59 (16820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:22:45 (16900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:47:03 (15540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:47:04 (15540): No heartbeat from core client for 30 sec - exiting 14:48:36 (14524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:49:21 (10656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:49 (16832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:12 (12360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:22:53 (18008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:27:25 (7060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:38:51 (11676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:58:40 (6744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:54 (7384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:08 (2500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:59:55 (5904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:49 (11916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:22 (8360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:26:16 (7384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:29:18 (14392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:01:18 (17944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:18:01 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:24:05 (5324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:28:38 (8276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:37:43 (13620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:16:11 (15896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:32:52 (13832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:51:01 (3696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:03:10 (12092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:33:29 (10284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:15 (8876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:00:46 (3996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:32:37 (6164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:08:56 (11716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:25:37 (17632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:39:14 (2088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:52:53 (1668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:35:32 (4684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:14:57 (5884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:30:07 (6436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:41:20 (12232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:51 (6056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:40:29 (11980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:12:17 (9960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:15:20 (10504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:24:25 (3360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:01 (15024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:42:35 (16504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:28:01 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:08:59 (11480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:34 (14700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:00:27 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:27:48 (3864): No heartbeat from core client for 30 sec - exiting 14:27:49 (3864): No heartbeat from core client for 30 sec - exiting 14:27:50 (3864): No heartbeat from core client for 30 sec - exiting 14:27:51 (3864): No heartbeat from core client for 30 sec - exiting 14:27:52 (3864): No heartbeat from core client for 30 sec - exiting 14:27:53 (3864): No heartbeat from core client for 30 sec - exiting 14:27:55 (3864): No heartbeat from core client for 30 sec - exiting 14:27:56 (3864): No heartbeat from core client for 30 sec - exiting 14:27:57 (3864): No heartbeat from core client for 30 sec - exiting 14:27:58 (3864): No heartbeat from core client for 30 sec - exiting 14:27:59 (3864): No heartbeat from core client for 30 sec - exiting 14:28:00 (3864): No heartbeat from core client for 30 sec - exiting 14:28:01 (3864): No heartbeat from core client for 30 sec - exiting 14:28:02 (3864): No heartbeat from core client for 30 sec - exiting 14:28:03 (3864): No heartbeat from core client for 30 sec - exiting 14:28:05 (3864): No heartbeat from core client for 30 sec - exiting 14:28:06 (3864): No heartbeat from core client for 30 sec - exiting 14:28:07 (3864): No heartbeat from core client for 30 sec - exiting 14:28:08 (3864): No heartbeat from core client for 30 sec - exiting 14:28:09 (3864): No heartbeat from core client for 30 sec - exiting 14:28:10 (3864): No heartbeat from core client for 30 sec - exiting 14:28:11 (3864): No heartbeat from core client for 30 sec - exiting 14:28:12 (3864): No heartbeat from core client for 30 sec - exiting 14:28:13 (3864): No heartbeat from core client for 30 sec - exiting 14:28:14 (3864): No heartbeat from core client for 30 sec - exiting 14:28:16 (3864): No heartbeat from core client for 30 sec - exiting 14:28:17 (3864): No heartbeat from core client for 30 sec - exiting 14:28:18 (3864): No heartbeat from core client for 30 sec - exiting 14:28:19 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2468, selfPID=2468, iMonCtr=1 No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3212, selfPID=3212, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7C910CCE read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Feb 2012 09:18:53	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	1,036,800	3,385,831	3.2657
24 Feb 2012 05:14:44	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	1,010,880	3,305,922	3.2703
22 Feb 2012 05:03:46	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	984,960	3,225,794	3.2751
15 Jan 2012 00:38:45	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	959,040	3,143,857	3.2781
13 Jan 2012 10:53:08	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	933,120	3,062,031	3.2815
06 Jan 2012 17:25:11	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	907,200	2,980,127	3.2850
28 Nov 2011 18:54:40	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	881,280	2,897,216	3.2875
09 Nov 2011 03:45:41	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	855,360	2,814,637	3.2906
08 Nov 2011 16:12:04	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	829,440	2,731,979	3.2938
08 Nov 2011 16:12:03	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	803,520	2,648,283	3.2959
08 Nov 2011 16:12:02	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	777,600	2,564,763	3.2983
08 Nov 2011 16:12:01	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	751,680	2,481,042	3.3007
08 Nov 2011 16:12:01	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	725,760	2,397,376	3.3033
31 Oct 2011 19:30:26	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	699,840	2,314,116	3.3066
31 Oct 2011 18:35:26	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	673,920	2,230,947	3.3104
31 Oct 2011 17:21:28	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	648,000	2,148,347	3.3154
31 Oct 2011 15:44:00	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	622,080	2,061,922	3.3146
30 Sep 2011 19:58:59	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	596,160	1,973,983	3.3112
14 Sep 2011 19:49:26	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	570,240	1,889,219	3.3130
13 Sep 2011 06:06:53	1090407	13144543	hadcm3n_yc9b_1900_40_007348745_2	544,320	1,804,899	3.3159