Task 13163261

Name	hadam3p_pnw_yzg6_1965_1_006910606_2
Workunit	7113922
Created	28 Jul 2011, 1:26:59 UTC
Sent	28 Jul 2011, 1:36:05 UTC
Report deadline	9 Jul 2012, 6:56:05 UTC
Received	11 Aug 2011, 10:01:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1102379
Run time	2 days 8 hours 24 min 31 sec
CPU time	2 days 2 hours 35 min 38 sec
Validate state	Invalid
Credit	1,253.67
Device peak FLOPS	2.94 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4304, selfPID=4224, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4720, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3892, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5060, selfPID=3516, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 20:53:17 (3968): No heartbeat from core client for 30 sec - exiting 20:53:18 (3968): No heartbeat from core client for 30 sec - exiting 20:53:19 (3968): No heartbeat from core client for 30 sec - exiting 20:53:20 (3968): No heartbeat from core client for 30 sec - exiting 20:53:21 (3968): No heartbeat from core client for 30 sec - exiting 20:53:22 (3968): No heartbeat from core client for 30 sec - exiting 20:53:23 (3968): No heartbeat from core client for 30 sec - exiting 20:53:24 (3968): No heartbeat from core client for 30 sec - exiting 20:53:25 (3968): No heartbeat from core client for 30 sec - exiting 20:53:26 (3968): No heartbeat from core client for 30 sec - exiting 20:53:27 (3968): No heartbeat from core client for 30 sec - exiting 20:53:28 (3968): No heartbeat from core client for 30 sec - exiting 20:53:29 (3968): No heartbeat from core client for 30 sec - exiting 20:53:30 (3968): No heartbeat from core client for 30 sec - exiting 20:53:31 (3968): No heartbeat from core client for 30 sec - exiting 20:53:32 (3968): No heartbeat from core client for 30 sec - exiting 20:53:33 (3968): No heartbeat from core client for 30 sec - exiting 20:53:34 (3968): No heartbeat from core client for 30 sec - exiting 20:53:35 (3968): No heartbeat from core client for 30 sec - exiting 20:53:36 (3968): No heartbeat from core client for 30 sec - exiting 20:53:37 (3968): No heartbeat from core client for 30 sec - exiting 20:53:38 (3968): No heartbeat from core client for 30 sec - exiting 20:53:39 (3968): No heartbeat from core client for 30 sec - exiting 20:53:40 (3968): No heartbeat from core client for 30 sec - exiting 20:53:41 (3968): No heartbeat from core client for 30 sec - exiting 20:53:42 (3968): No heartbeat from core client for 30 sec - exiting 20:53:43 (3968): No heartbeat from core client for 30 sec - exiting 20:53:44 (3968): No heartbeat from core client for 30 sec - exiting 20:53:45 (3968): No heartbeat from core client for 30 sec - exiting 20:53:46 (3968): No heartbeat from core client for 30 sec - exiting 20:53:47 (3968): No heartbeat from core client for 30 sec - exiting 20:53:48 (3968): No heartbeat from core client for 30 sec - exiting 20:53:49 (3968): No heartbeat from core client for 30 sec - exiting 20:53:50 (3968): No heartbeat from core client for 30 sec - exiting 20:53:51 (3968): No heartbeat from core client for 30 sec - exiting 20:53:52 (3968): No heartbeat from core client for 30 sec - exiting 20:53:53 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:53:54 (3968): No heartbeat from core client for 30 sec - exiting 20:12:10 (3452): No heartbeat from core client for 30 sec - exiting 20:12:11 (3452): No heartbeat from core client for 30 sec - exiting 20:12:12 (3452): No heartbeat from core client for 30 sec - exiting 20:12:13 (3452): No heartbeat from core client for 30 sec - exiting 20:12:14 (3452): No heartbeat from core client for 30 sec - exiting 20:12:15 (3452): No heartbeat from core client for 30 sec - exiting 20:12:16 (3452): No heartbeat from core client for 30 sec - exiting 20:12:17 (3452): No heartbeat from core client for 30 sec - exiting 20:12:18 (3452): No heartbeat from core client for 30 sec - exiting 20:12:19 (3452): No heartbeat from core client for 30 sec - exiting 20:12:20 (3452): No heartbeat from core client for 30 sec - exiting 20:12:21 (3452): No heartbeat from core client for 30 sec - exiting 20:12:22 (3452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:12:23 (3452): No heartbeat from core client for 30 sec - exiting 07:31:31 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:02 (4864): No heartbeat from core client for 30 sec - exiting 23:24:03 (4864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:47 (4060): No heartbeat from core client for 30 sec - exiting 17:45:48 (4060): No heartbeat from core client for 30 sec - exiting 17:45:49 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3420, selfPID=5100, iMonCtr=1 Model crash detected, will try to restart... 22:24:56 (2972): No heartbeat from core client for 30 sec - exiting 22:24:58 (2972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4864, iMonCtr=2 Model crash detected, will try to restart... 18:18:37 (3224): No heartbeat from core client for 30 sec - exiting 18:18:38 (3224): No heartbeat from core client for 30 sec - exiting 18:18:39 (3224): No heartbeat from core client for 30 sec - exiting 18:18:40 (3224): No heartbeat from core client for 30 sec - exiting 18:18:41 (3224): No heartbeat from core client for 30 sec - exiting 18:18:42 (3224): No heartbeat from core client for 30 sec - exiting 18:18:43 (3224): No heartbeat from core client for 30 sec - exiting 18:18:44 (3224): No heartbeat from core client for 30 sec - exiting 18:18:45 (3224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:18:46 (3224): No heartbeat from core client for 30 sec - exiting 20:48:24 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2556, selfPID=1428, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:10:02 (1084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:03 (1084): No heartbeat from core client for 30 sec - exiting 08:10:04 (1084): No heartbeat from core client for 30 sec - exiting 08:10:05 (1084): No heartbeat from core client for 30 sec - exiting 08:10:06 (1084): No heartbeat from core client for 30 sec - exiting 08:10:07 (1084): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4020, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4028, selfPID=3904, iMonCtr=1 Model crash detected, will try to restart... 09:13:33 (1848): No heartbeat from core client for 30 sec - exiting 09:13:34 (1848): No heartbeat from core client for 30 sec - exiting 09:13:35 (1848): No heartbeat from core client for 30 sec - exiting 09:13:36 (1848): No heartbeat from core client for 30 sec - exiting 09:13:37 (1848): No heartbeat from core client for 30 sec - exiting 09:13:38 (1848): No heartbeat from core client for 30 sec - exiting 09:13:39 (1848): No heartbeat from core client for 30 sec - exiting 09:13:40 (1848): No heartbeat from core client for 30 sec - exiting 09:13:41 (1848): No heartbeat from core client for 30 sec - exiting 09:13:42 (1848): No heartbeat from core client for 30 sec - exiting 09:13:43 (1848): No heartbeat from core client for 30 sec - exiting 09:13:44 (1848): No heartbeat from core client for 30 sec - exiting 09:13:45 (1848): No heartbeat from core client for 30 sec - exiting 09:13:46 (1848): No heartbeat from core client for 30 sec - exiting 09:13:47 (1848): No heartbeat from core client for 30 sec - exiting 09:13:48 (1848): No heartbeat from core client for 30 sec - exiting 09:13:49 (1848): No heartbeat from core client for 30 sec - exiting 09:13:50 (1848): No heartbeat from core client for 30 sec - exiting 09:13:51 (1848): No heartbeat from core client for 30 sec - exiting 09:13:52 (1848): No heartbeat from core client for 30 sec - exiting 09:13:53 (1848): No heartbeat from core client for 30 sec - exiting 09:13:54 (1848): No heartbeat from core client for 30 sec - exiting 09:13:55 (1848): No heartbeat from core client for 30 sec - exiting 09:13:56 (1848): No heartbeat from core client for 30 sec - exiting 09:13:57 (1848): No heartbeat from core client for 30 sec - exiting 09:13:58 (1848): No heartbeat from core client for 30 sec - exiting 09:13:59 (1848): No heartbeat from core client for 30 sec - exiting 09:14:00 (1848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:14:01 (1848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4480, selfPID=4480, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4544, selfPID=4544, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=792, selfPID=792, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=160, selfPID=160, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2496, selfPID=2496, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4916, selfPID=4916, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=672, selfPID=672, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4916, selfPID=4916, iMonCtr=2 CPDN Monitor - Quit request from BOINC... No ProcesCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=876, selfPID=876, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle RCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4524, selfPID=4524, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4680, selfPID=4680, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4388, selfPID=4388, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3984, selfPID=3984, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=620, selfPID=620, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3948, selfPID=3948, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3848, selfPID=3848, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process HanCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2704, selfPID=2704, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2732, selfPID=2732, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=472, selfPID=472, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=132, selfPID=132, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2768, selfPID=2768, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3076, selfPID=3076, iMonCtr=2 CPDN Monitor - Quit request from BOINC... No PrCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4756, selfPID=4756, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3632, selfPID=3632, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1168, selfPID=1168, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2536, selfPID=2536, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4256, selfPID=4256, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3168, selfPID=3168, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4612, selfPID=4612, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3880, selfPID=3880, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 5 Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_pnw_yzg6_1965_1_006910606_2_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_yzg6_1965_1_006910606_2_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_yzg6_1965_1_006910606_2_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_yzg6_1965_1_006910606_2_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_yzg6_1965_1_006910606_2_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_yzg6_1965_1_006910606_2_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_yzg6_1965_1_006910606_2_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Aug 2011 10:03:10	1102379	13163261	hadam3p_pnw_yzg6_1965_1_006910606_2	57,696	180,188	3.1231
08 Aug 2011 03:50:22	1102379	13163261	hadam3p_pnw_yzg6_1965_1_006910606_2	46,176	145,188	3.1442
04 Aug 2011 11:20:41	1102379	13163261	hadam3p_pnw_yzg6_1965_1_006910606_2	34,656	110,200	3.1798
31 Jul 2011 13:03:59	1102379	13163261	hadam3p_pnw_yzg6_1965_1_006910606_2	23,136	74,423	3.2168
29 Jul 2011 08:58:31	1102379	13163261	hadam3p_pnw_yzg6_1965_1_006910606_2	11,616	37,920	3.2645