Task 12769176

Name	hadcm3n_o0r8_1900_40_007196311_2
Workunit	7394591
Created	2 Apr 2011, 19:05:19 UTC
Sent	2 Apr 2011, 19:23:17 UTC
Report deadline	3 Jul 2011, 2:50:28 UTC
Received	9 Jun 2011, 6:49:36 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1125581
Run time	29 days 6 hours 27 min 19 sec
CPU time	20 days 7 hours 47 min 41 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:53:30 (7144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:58:46 (5496): No heartbeat from core client for 30 sec - exiting 15:58:47 (5496): No heartbeat from core client for 30 sec - exiting 15:58:48 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6388, iMonCtr=1 Model crash detected, will try to restart... 14:45:02 (2464): No heartbeat from core client for 30 sec - exiting 14:45:03 (2464): No heartbeat from core client for 30 sec - exiting 14:45:04 (2464): No heartbeat from core client for 30 sec - exiting 14:45:05 (2464): No heartbeat from core client for 30 sec - exiting 14:45:06 (2464): No heartbeat from core client for 30 sec - exiting 14:45:07 (2464): No heartbeat from core client for 30 sec - exiting 14:45:08 (2464): No heartbeat from core client for 30 sec - exiting 14:45:09 (2464): No heartbeat from core client for 30 sec - exiting 14:45:10 (2464): No heartbeat from core client for 30 sec - exiting 14:45:11 (2464): No heartbeat from core client for 30 sec - exiting 14:45:12 (2464): No heartbeat from core client for 30 sec - exiting 14:45:13 (2464): No heartbeat from core client for 30 sec - exiting 14:45:14 (2464): No heartbeat from core client for 30 sec - exiting 14:45:15 (2464): No heartbeat from core client for 30 sec - exiting 14:45:16 (2464): No heartbeat from core client for 30 sec - exiting 14:45:17 (2464): No heartbeat from core client for 30 sec - exiting 14:45:18 (2464): No heartbeat from core client for 30 sec - exiting 14:45:19 (2464): No heartbeat from core client for 30 sec - exiting 14:45:20 (2464): No heartbeat from core client for 30 sec - exiting 14:45:21 (2464): No heartbeat from core client for 30 sec - exiting 14:45:22 (2464): No heartbeat from core client for 30 sec - exiting 14:45:23 (2464): No heartbeat from core client for 30 sec - exiting 14:45:24 (2464): No heartbeat from core client for 30 sec - exiting 14:45:25 (2464): No heartbeat from core client for 30 sec - exiting 14:45:26 (2464): No heartbeat from core client for 30 sec - exiting 14:45:27 (2464): No heartbeat from core client for 30 sec - exiting 14:45:28 (2464): No heartbeat from core client for 30 sec - exiting 14:45:29 (2464): No heartbeat from core client for 30 sec - exiting 14:45:30 (2464): No heartbeat from core client for 30 sec - exiting 14:45:31 (2464): No heartbeat from core client for 30 sec - exiting 14:45:32 (2464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:18:47 (6816): No heartbeat from core client for 30 sec - exiting 10:18:48 (6816): No heartbeat from core client for 30 sec - exiting 10:18:50 (6816): No heartbeat from core client for 30 sec - exiting 10:18:51 (6816): No heartbeat from core client for 30 sec - exiting 10:18:52 (6816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:22:41 (5248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 03:15:09 (12936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:53:48 (13304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:18 (880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:42:31 (14232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:54:38 (13604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:40:47 (3816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:58 (3816): No heartbeat from core client for 30 sec - exiting 23:40:59 (3816): No heartbeat from core client for 30 sec - exiting 23:41:00 (3816): No heartbeat from core client for 30 sec - exiting 23:43:01 (3292): No heartbeat from core client for 30 sec - exiting 23:43:02 (3292): No heartbeat from core client for 30 sec - exiting 23:43:03 (3292): No heartbeat from core client for 30 sec - exiting 23:43:04 (3292): No heartbeat from core client for 30 sec - exiting 23:43:05 (3292): No heartbeat from core client for 30 sec - exiting 23:43:06 (3292): No heartbeat from core client for 30 sec - exiting 23:43:07 (3292): No heartbeat from core client for 30 sec - exiting 23:43:08 (3292): No heartbeat from core client for 30 sec - exiting 23:43:09 (3292): No heartbeat from core client for 30 sec - exiting 23:43:10 (3292): No heartbeat from core client for 30 sec - exiting 23:43:11 (3292): No heartbeat from core client for 30 sec - exiting 23:43:12 (3292): No heartbeat from core client for 30 sec - exiting 23:43:13 (3292): No heartbeat from core client for 30 sec - exiting 23:43:14 (3292): No heartbeat from core client for 30 sec - exiting 23:43:15 (3292): No heartbeat from core client for 30 sec - exiting 23:43:16 (3292): No heartbeat from core client for 30 sec - exiting 23:43:17 (3292): No heartbeat from core client for 30 sec - exiting 23:43:18 (3292): No heartbeat from core client for 30 sec - exiting 23:43:19 (3292): No heartbeat from core client for 30 sec - exiting 23:43:20 (3292): No heartbeat from core client for 30 sec - exiting 23:43:21 (3292): No heartbeat from core client for 30 sec - exiting 23:43:22 (3292): No heartbeat from core client for 30 sec - exiting 23:43:23 (3292): No heartbeat from core client for 30 sec - exiting 23:43:24 (3292): No heartbeat from core client for 30 sec - exiting 23:43:25 (3292): No heartbeat from core client for 30 sec - exiting 23:43:26 (3292): No heartbeat from core client for 30 sec - exiting 23:43:27 (3292): No heartbeat from core client for 30 sec - exiting 23:43:28 (3292): No heartbeat from core client for 30 sec - exiting 23:43:29 (3292): No heartbeat from core client for 30 sec - exiting 23:43:30 (3292): No heartbeat from core client for 30 sec - exiting 23:43:31 (3292): No heartbeat from core client for 30 sec - exiting 23:43:32 (3292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:42:22 (12220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:42:57 (3096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:01:20 (9568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:36:39 (12776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:37:32 (13028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:13:42 (3004): No heartbeat from core client for 30 sec - exiting 16:13:43 (3004): No heartbeat from core client for 30 sec - exiting 16:13:44 (3004): No heartbeat from core client for 30 sec - exiting 16:13:45 (3004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:16:38 (11764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:06:35 (10976): No heartbeat from core client for 30 sec - exiting 00:06:36 (10976): No heartbeat from core client for 30 sec - exiting 00:06:37 (10976): No heartbeat from core client for 30 sec - exiting 00:06:38 (10976): No heartbeat from core client for 30 sec - exiting 00:06:39 (10976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:34:19 (6672): No heartbeat from core client for 30 sec - exiting 03:34:20 (6672): No heartbeat from core client for 30 sec - exiting 03:34:21 (6672): No heartbeat from core client for 30 sec - exiting 03:34:22 (6672): No heartbeat from core client for 30 sec - exiting 03:34:23 (6672): No heartbeat from core client for 30 sec - exiting 03:34:24 (6672): No heartbeat from core client for 30 sec - exiting 03:34:25 (6672): No heartbeat from core client for 30 sec - exiting 03:34:26 (6672): No heartbeat from core client for 30 sec - exiting 03:34:27 (6672): No heartbeat from core client for 30 sec - exiting 03:34:29 (6672): No heartbeat from core client for 30 sec - exiting 03:34:30 (6672): No heartbeat from core client for 30 sec - exiting 03:34:31 (6672): No heartbeat from core client for 30 sec - exiting 03:34:32 (6672): No heartbeat from core client for 30 sec - exiting 03:34:33 (6672): No heartbeat from core client for 30 sec - exiting 03:34:34 (6672): No heartbeat from core client for 30 sec - exiting 03:34:35 (6672): No heartbeat from core client for 30 sec - exiting 03:34:36 (6672): No heartbeat from core client for 30 sec - exiting 03:34:37 (6672): No heartbeat from core client for 30 sec - exiting 03:34:38 (6672): No heartbeat from core client for 30 sec - exiting 03:34:39 (6672): No heartbeat from core client for 30 sec - exiting 03:34:41 (6672): No heartbeat from core client for 30 sec - exiting 03:34:42 (6672): No heartbeat from core client for 30 sec - exiting 03:34:43 (6672): No heartbeat from core client for 30 sec - exiting 03:34:44 (6672): No heartbeat from core client for 30 sec - exiting 03:34:45 (6672): No heartbeat from core client for 30 sec - exiting 03:34:46 (6672): No heartbeat from core client for 30 sec - exiting 03:34:47 (6672): No heartbeat from core client for 30 sec - exiting 03:34:48 (6672): No heartbeat from core client for 30 sec - exiting 03:34:50 (6672): No heartbeat from core client for 30 sec - exiting 03:34:51 (6672): No heartbeat from core client for 30 sec - exiting 03:34:52 (6672): No heartbeat from core client for 30 sec - exiting 03:34:53 (6672): No heartbeat from core client for 30 sec - exiting 03:34:54 (6672): No heartbeat from core client for 30 sec - exiting 03:34:55 (6672): No heartbeat from core client for 30 sec - exiting 03:34:56 (6672): No heartbeat from core client for 30 sec - exiting 03:34:57 (6672): No heartbeat from core client for 30 sec - exiting 03:34:58 (6672): No heartbeat from core client for 30 sec - exiting 03:34:59 (6672): No heartbeat from core client for 30 sec - exiting 03:35:00 (6672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:56:28 (4812): No heartbeat from core client for 30 sec - exiting 05:56:29 (4812): No heartbeat from core client for 30 sec - exiting 05:56:30 (4812): No heartbeat from core client for 30 sec - exiting 05:56:31 (4812): No heartbeat from core client for 30 sec - exiting 05:56:32 (4812): No heartbeat from core client for 30 sec - exiting 05:56:33 (4812): No heartbeat from core client for 30 sec - exiting 05:56:34 (4812): No heartbeat from core client for 30 sec - exiting 05:56:35 (4812): No heartbeat from core client for 30 sec - exiting 05:56:36 (4812): No heartbeat from core client for 30 sec - exiting 05:56:37 (4812): No heartbeat from core client for 30 sec - exiting 05:56:38 (4812): No heartbeat from core client for 30 sec - exiting 05:56:39 (4812): No heartbeat from core client for 30 sec - exiting 05:56:40 (4812): No heartbeat from core client for 30 sec - exiting 05:56:41 (4812): No heartbeat from core client for 30 sec - exiting 05:56:42 (4812): No heartbeat from core client for 30 sec - exiting 05:56:43 (4812): No heartbeat from core client for 30 sec - exiting 05:56:44 (4812): No heartbeat from core client for 30 sec - exiting 05:56:45 (4812): No heartbeat from core client for 30 sec - exiting 05:56:46 (4812): No heartbeat from core client for 30 sec - exiting 05:56:47 (4812): No heartbeat from core client for 30 sec - exiting 05:56:48 (4812): No heartbeat from core client for 30 sec - exiting 05:56:49 (4812): No heartbeat from core client for 30 sec - exiting 05:56:50 (4812): No heartbeat from core client for 30 sec - exiting 05:56:51 (4812): No heartbeat from core client for 30 sec - exiting 05:56:52 (4812): No heartbeat from core client for 30 sec - exiting 05:56:53 (4812): No heartbeat from core client for 30 sec - exiting 05:56:54 (4812): No heartbeat from core client for 30 sec - exiting 05:56:55 (4812): No heartbeat from core client for 30 sec - exiting 05:56:56 (4812): No heartbeat from core client for 30 sec - exiting 05:56:57 (4812): No heartbeat from core client for 30 sec - exiting 05:56:58 (4812): No heartbeat from core client for 30 sec - exiting 05:56:59 (4812): No heartbeat from core client for 30 sec - exiting 05:57:00 (4812): No heartbeat from core client for 30 sec - exiting 05:57:01 (4812): No heartbeat from core client for 30 sec - exiting 05:57:02 (4812): No heartbeat from core client for 30 sec - exiting 05:57:03 (4812): No heartbeat from core client for 30 sec - exiting 05:57:04 (4812): No heartbeat from core client for 30 sec - exiting 05:57:05 (4812): No heartbeat from core client for 30 sec - exiting 05:57:06 (4812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:59:50 (7552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7356, selfPID=7356, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:59:56 (6444): No heartbeat from core client for 30 sec - exiting 17:59:57 (6444): No heartbeat from core client for 30 sec - exiting 17:59:58 (6444): No heartbeat from core client for 30 sec - exiting 17:59:59 (6444): No heartbeat from core client for 30 sec - exiting 18:00:00 (6444): No heartbeat from core client for 30 sec - exiting 18:00:01 (6444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:32:00 (9380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:37:46 (9688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:26:34 (9148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:49:15 (10180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:49:23 (10180): No heartbeat from core client for 30 sec - exiting 08:49:24 (10180): No heartbeat from core client for 30 sec - exiting 08:49:25 (10180): No heartbeat from core client for 30 sec - exiting 08:49:26 (10180): No heartbeat from core client for 30 sec - exiting 08:49:27 (10180): No heartbeat from core client for 30 sec - exiting 08:49:28 (10180): No heartbeat from core client for 30 sec - exiting 08:49:29 (10180): No heartbeat from core client for 30 sec - exiting 08:49:30 (10180): No heartbeat from core client for 30 sec - exiting 08:49:31 (10180): No heartbeat from core client for 30 sec - exiting 08:49:32 (10180): No heartbeat from core client for 30 sec - exiting 08:49:33 (10180): No heartbeat from core client for 30 sec - exiting 08:49:34 (10180): No heartbeat from core client for 30 sec - exiting 08:49:35 (10180): No heartbeat from core client for 30 sec - exiting 08:49:36 (10180): No heartbeat from core client for 30 sec - exiting 08:49:37 (10180): No heartbeat from core client for 30 sec - exiting 08:49:38 (10180): No heartbeat from core client for 30 sec - exiting 08:49:39 (10180): No heartbeat from core client for 30 sec - exiting 08:49:40 (10180): No heartbeat from core client for 30 sec - exiting 08:49:41 (10180): No heartbeat from core client for 30 sec - exiting 08:49:42 (10180): No heartbeat from core client for 30 sec - exiting 08:49:43 (10180): No heartbeat from core client for 30 sec - exiting 08:49:44 (10180): No heartbeat from core client for 30 sec - exiting 08:49:45 (10180): No heartbeat from core client for 30 sec - exiting 08:49:46 (10180): No heartbeat from core client for 30 sec - exiting 08:49:47 (10180): No heartbeat from core client for 30 sec - exiting 08:49:48 (10180): No heartbeat from core client for 30 sec - exiting 08:49:49 (10180): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 10:47:10 (8432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:47:15 (8432): No heartbeat from core client for 30 sec - exiting 10:47:16 (8432): No heartbeat from core client for 30 sec - exiting 10:47:17 (8432): No heartbeat from core client for 30 sec - exiting 10:47:18 (8432): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... 20:09:21 (3232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:22 (6252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:10 (5976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:52:09 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:57:02 (7720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:08:30 (3744): No heartbeat from core client for 30 sec - exiting 17:08:31 (3744): No heartbeat from core client for 30 sec - exiting 17:08:33 (3744): No heartbeat from core client for 30 sec - exiting 17:08:34 (3744): No heartbeat from core client for 30 sec - exiting 17:08:35 (3744): No heartbeat from core client for 30 sec - exiting 17:08:36 (3744): No heartbeat from core client for 30 sec - exiting 17:08:37 (3744): No heartbeat from core client for 30 sec - exiting 17:08:38 (3744): No heartbeat from core client for 30 sec - exiting 17:08:39 (3744): No heartbeat from core client for 30 sec - exiting 17:08:40 (3744): No heartbeat from core client for 30 sec - exiting 17:08:41 (3744): No heartbeat from core client for 30 sec - exiting 17:08:42 (3744): No heartbeat from core client for 30 sec - exiting 17:08:43 (3744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:08:44 (3744): No heartbeat from core client for 30 sec - exiting 17:08:45 (3744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o0r8ko.dae0c20 CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Jun 2011 04:19:10	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	1,036,800	1,755,396	1.6931
08 Jun 2011 09:31:20	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	1,010,880	1,710,124	1.6917
07 Jun 2011 08:54:29	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	984,960	1,667,647	1.6931
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	959,040	1,624,927	1.6943
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	933,120	1,582,738	1.6962
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	907,200	1,540,459	1.6980
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	881,280	1,498,447	1.7003
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	855,360	1,457,099	1.7035
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	829,440	1,415,190	1.7062
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	803,520	1,373,213	1.7090
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	777,600	1,331,237	1.7120
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	751,680	1,289,252	1.7152
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	725,760	1,246,837	1.7180
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	699,840	1,203,079	1.7191
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	673,920	1,158,087	1.7184
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	648,000	1,113,124	1.7178
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	622,080	1,068,480	1.7176
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	596,160	1,024,309	1.7182
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	570,240	980,772	1.7199
07 Jun 2011 01:44:31	1125581	12769176	hadcm3n_o0r8_1900_40_007196311_2	544,320	937,649	1.7226