Task 15790824

Name	hadcm3n_zazf_1960_40_008334019_3
Workunit	8484880
Created	20 May 2013, 14:58:38 UTC
Sent	20 May 2013, 14:58:40 UTC
Report deadline	19 Aug 2013, 22:25:51 UTC
Received	7 Jun 2013, 23:23:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1143449
Run time	5 days 20 hours 16 min 35 sec
CPU time	5 days 14 hours 1 min 42 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.02 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9684, iMonCtr=1 Model crash detected, will try to restart... 22:11:37 (6600): No heartbeat from core client for 30 sec - exiting 22:11:38 (6600): No heartbeat from core client for 30 sec - exiting 22:11:39 (6600): No heartbeat from core client for 30 sec - exiting 22:11:40 (6600): No heartbeat from core client for 30 sec - exiting 22:11:41 (6600): No heartbeat from core client for 30 sec - exiting 22:11:42 (6600): No heartbeat from core client for 30 sec - exiting 22:11:43 (6600): No heartbeat from core client for 30 sec - exiting 22:11:44 (6600): No heartbeat from core client for 30 sec - exiting 22:11:45 (6600): No heartbeat from core client for 30 sec - exiting 22:11:46 (6600): No heartbeat from core client for 30 sec - exiting 22:11:47 (6600): No heartbeat from core client for 30 sec - exiting 22:11:48 (6600): No heartbeat from core client for 30 sec - exiting 22:11:49 (6600): No heartbeat from core client for 30 sec - exiting 22:11:50 (6600): No heartbeat from core client for 30 sec - exiting 22:11:51 (6600): No heartbeat from core client for 30 sec - exiting 22:11:52 (6600): No heartbeat from core client for 30 sec - exiting 22:11:53 (6600): No heartbeat from core client for 30 sec - exiting 22:11:54 (6600): No heartbeat from core client for 30 sec - exiting 22:11:55 (6600): No heartbeat from core client for 30 sec - exiting 22:11:56 (6600): No heartbeat from core client for 30 sec - exiting 22:11:57 (6600): No heartbeat from core client for 30 sec - exiting 22:11:58 (6600): No heartbeat from core client for 30 sec - exiting 22:11:59 (6600): No heartbeat from core client for 30 sec - exiting 22:12:00 (6600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4292, iMonCtr=1 Model crash detected, will try to restart... 11:41:11 (6184): No heartbeat from core client for 30 sec - exiting 11:41:12 (6184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6836, iMonCtr=1 Model crash detected, will try to restart... 11:14:19 (6196): No heartbeat from core client for 30 sec - exiting 11:14:20 (6196): No heartbeat from core client for 30 sec - exiting 11:14:21 (6196): No heartbeat from core client for 30 sec - exiting 11:14:22 (6196): No heartbeat from core client for 30 sec - exiting 11:14:23 (6196): No heartbeat from core client for 30 sec - exiting 11:14:24 (6196): No heartbeat from core client for 30 sec - exiting 11:14:25 (6196): No heartbeat from core client for 30 sec - exiting 11:14:26 (6196): No heartbeat from core client for 30 sec - exiting 11:14:27 (6196): No heartbeat from core client for 30 sec - exiting 11:14:28 (6196): No heartbeat from core client for 30 sec - exiting 11:14:30 (6196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7432, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6080, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7832, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6352, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6352, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6352, iMonCtr=1 Model crash detected, will try to restart... 10:49:09 (6440): No heartbeat from core client for 30 sec - exiting 10:49:10 (6440): No heartbeat from core client for 30 sec - exiting 10:49:11 (6440): No heartbeat from core client for 30 sec - exiting 10:49:12 (6440): No heartbeat from core client for 30 sec - exiting 10:49:13 (6440): No heartbeat from core client for 30 sec - exiting 10:49:14 (6440): No heartbeat from core client for 30 sec - exiting 10:49:15 (6440): No heartbeat from core client for 30 sec - exiting 10:49:16 (6440): No heartbeat from core client for 30 sec - exiting 10:49:17 (6440): No heartbeat from core client for 30 sec - exiting 10:49:18 (6440): No heartbeat from core client for 30 sec - exiting 10:49:19 (6440): No heartbeat from core client for 30 sec - exiting 10:49:20 (6440): No heartbeat from core client for 30 sec - exiting 10:49:21 (6440): No heartbeat from core client for 30 sec - exiting 10:49:22 (6440): No heartbeat from core client for 30 sec - exiting 10:49:23 (6440): No heartbeat from core client for 30 sec - exiting 10:49:24 (6440): No heartbeat from core client for 30 sec - exiting 10:49:25 (6440): No heartbeat from core client for 30 sec - exiting 10:49:26 (6440): No heartbeat from core client for 30 sec - exiting 10:49:27 (6440): No heartbeat from core client for 30 sec - exiting 10:49:28 (6440): No heartbeat from core client for 30 sec - exiting 10:49:29 (6440): No heartbeat from core client for 30 sec - exiting 10:49:30 (6440): No heartbeat from core client for 30 sec - exiting 10:49:31 (6440): No heartbeat from core client for 30 sec - exiting 10:49:32 (6440): No heartbeat from core client for 30 sec - exiting 10:49:33 (6440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:35:49 (6368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8160, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8120, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8120, iMonCtr=1 Model crash detected, will try to restart... 21:37:40 (6788): No heartbeat from core client for 30 sec - exiting 21:37:41 (6788): No heartbeat from core client for 30 sec - exiting 21:37:42 (6788): No heartbeat from core client for 30 sec - exiting 21:37:43 (6788): No heartbeat from core client for 30 sec - exiting 21:37:44 (6788): No heartbeat from core client for 30 sec - exiting 21:37:45 (6788): No heartbeat from core client for 30 sec - exiting 21:37:46 (6788): No heartbeat from core client for 30 sec - exiting 21:37:47 (6788): No heartbeat from core client for 30 sec - exiting 21:37:48 (6788): No heartbeat from core client for 30 sec - exiting 21:37:49 (6788): No heartbeat from core client for 30 sec - exiting 21:37:50 (6788): No heartbeat from core client for 30 sec - exiting 21:37:51 (6788): No heartbeat from core client for 30 sec - exiting 21:37:52 (6788): No heartbeat from core client for 30 sec - exiting 21:37:53 (6788): No heartbeat from core client for 30 sec - exiting 21:37:54 (6788): No heartbeat from core client for 30 sec - exiting 21:37:55 (6788): No heartbeat from core client for 30 sec - exiting 21:37:56 (6788): No heartbeat from core client for 30 sec - exiting 21:37:57 (6788): No heartbeat from core client for 30 sec - exiting 21:37:58 (6788): No heartbeat from core client for 30 sec - exiting 21:37:59 (6788): No heartbeat from core client for 30 sec - exiting 21:38:00 (6788): No heartbeat from core client for 30 sec - exiting 21:38:01 (6788): No heartbeat from core client for 30 sec - exiting 21:38:02 (6788): No heartbeat from core client for 30 sec - exiting 21:38:03 (6788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7684, iMonCtr=1 Model crash detected, will try to restart... 07:52:50 (6756): No heartbeat from core client for 30 sec - exiting 07:52:51 (6756): No heartbeat from core client for 30 sec - exiting 07:52:52 (6756): No heartbeat from core client for 30 sec - exiting 07:52:53 (6756): No heartbeat from core client for 30 sec - exiting 07:52:54 (6756): No heartbeat from core client for 30 sec - exiting 07:52:55 (6756): No heartbeat from core client for 30 sec - exiting 07:52:56 (6756): No heartbeat from core client for 30 sec - exiting 07:52:57 (6756): No heartbeat from core client for 30 sec - exiting 07:52:58 (6756): No heartbeat from core client for 30 sec - exiting 07:52:59 (6756): No heartbeat from core client for 30 sec - exiting 07:53:00 (6756): No heartbeat from core client for 30 sec - exiting 07:53:01 (6756): No heartbeat from core client for 30 sec - exiting 07:53:02 (6756): No heartbeat from core client for 30 sec - exiting 07:53:03 (6756): No heartbeat from core client for 30 sec - exiting 07:53:04 (6756): No heartbeat from core client for 30 sec - exiting 07:53:05 (6756): No heartbeat from core client for 30 sec - exiting 07:53:06 (6756): No heartbeat from core client for 30 sec - exiting 07:53:07 (6756): No heartbeat from core client for 30 sec - exiting 07:53:08 (6756): No heartbeat from core client for 30 sec - exiting 07:53:09 (6756): No heartbeat from core client for 30 sec - exiting 07:53:10 (6756): No heartbeat from core client for 30 sec - exiting 07:53:11 (6756): No heartbeat from core client for 30 sec - exiting 07:53:12 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:30:13 (6644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5080, iMonCtr=1 Model crash detected, will try to restart... 11:01:26 (6688): No heartbeat from core client for 30 sec - exiting 11:01:27 (6688): No heartbeat from core client for 30 sec - exiting 11:01:28 (6688): No heartbeat from core client for 30 sec - exiting 11:01:29 (6688): No heartbeat from core client for 30 sec - exiting 11:01:30 (6688): No heartbeat from core client for 30 sec - exiting 11:01:31 (6688): No heartbeat from core client for 30 sec - exiting 11:01:32 (6688): No heartbeat from core client for 30 sec - exiting 11:01:33 (6688): No heartbeat from core client for 30 sec - exiting 11:01:34 (6688): No heartbeat from core client for 30 sec - exiting 11:01:35 (6688): No heartbeat from core client for 30 sec - exiting 11:01:36 (6688): No heartbeat from core client for 30 sec - exiting 11:01:37 (6688): No heartbeat from core client for 30 sec - exiting 11:01:38 (6688): No heartbeat from core client for 30 sec - exiting 11:01:39 (6688): No heartbeat from core client for 30 sec - exiting 11:01:40 (6688): No heartbeat from core client for 30 sec - exiting 11:01:41 (6688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:01:42 (6688): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7716, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7412, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7412, iMonCtr=1 Model crash detected, will try to restart... 11:14:35 (7360): No heartbeat from core client for 30 sec - exiting 11:14:36 (7360): No heartbeat from core client for 30 sec - exiting 11:14:37 (7360): No heartbeat from core client for 30 sec - exiting 11:14:38 (7360): No heartbeat from core client for 30 sec - exiting 11:14:39 (7360): No heartbeat from core client for 30 sec - exiting 11:14:40 (7360): No heartbeat from core client for 30 sec - exiting 11:14:41 (7360): No heartbeat from core client for 30 sec - exiting 11:14:42 (7360): No heartbeat from core client for 30 sec - exiting 11:14:43 (7360): No heartbeat from core client for 30 sec - exiting 11:14:44 (7360): No heartbeat from core client for 30 sec - exiting 11:14:45 (7360): No heartbeat from core client for 30 sec - exiting 11:14:46 (7360): No heartbeat from core client for 30 sec - exiting 11:14:47 (7360): No heartbeat from core client for 30 sec - exiting 11:14:48 (7360): No heartbeat from core client for 30 sec - exiting 11:14:49 (7360): No heartbeat from core client for 30 sec - exiting 11:14:50 (7360): No heartbeat from core client for 30 sec - exiting 11:14:51 (7360): No heartbeat from core client for 30 sec - exiting 11:14:52 (7360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2968, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 10:30:21 (7408): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4856, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C5C3EB write attempt to address 0xC2384FE4 Engaging BOINC Windows Runtime Debugger... 00:02:38 (6628): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A55EAB read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Jun 2013 20:21:09	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	259,200	473,350	1.8262
06 Jun 2013 07:20:34	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	233,280	425,551	1.8242
05 Jun 2013 17:53:16	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	207,360	378,688	1.8262
04 Jun 2013 17:49:10	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	181,440	331,202	1.8254
03 Jun 2013 11:56:05	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	155,520	283,456	1.8226
02 Jun 2013 09:03:51	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	129,600	236,163	1.8222
28 May 2013 10:54:28	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	103,680	188,546	1.8185
26 May 2013 18:18:25	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	77,760	140,938	1.8125
24 May 2013 12:14:00	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	51,840	93,473	1.8031
22 May 2013 16:01:18	1143449	15790824	hadcm3n_zazf_1960_40_008334019_3	25,920	47,457	1.8309