Task 12756099

Name	hadam3p_pnw_z4ol_1980_1_006926989_2
Workunit	7130305
Created	30 Mar 2011, 1:41:35 UTC
Sent	30 Mar 2011, 7:37:10 UTC
Report deadline	11 Mar 2012, 12:57:10 UTC
Received	25 May 2011, 9:39:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1100062
Run time	4 days 11 hours 40 min 50 sec
CPU time	4 days 10 hours 43 min 55 sec
Validate state	Invalid
Credit	1,253.67
Device peak FLOPS	1.65 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <stderr_txt> error: cannot delete old C:/Dokumente und Einstellungen/All Users/Anwendungsdaten/BOINC/projects/climateprediction.net/txf/Helvetica-Bold.txf error: cannot delete old C:/Dokumente und Einstellungen/All Users/Anwendungsdaten/BOINC/projects/climateprediction.net/txf/Times-Roman.txf error: cannot delete old C:/Dokumente und Einstellungen/All Users/Anwendungsdaten/BOINC/projects/climateprediction.net/txf/Times-Italic.txf error: cannot delete old C:/Dokumente und Einstellungen/All Users/Anwendungsdaten/BOINC/projects/climateprediction.net/txf/Times-Bold.txf error: cannot delete old C:/Dokumente und Einstellungen/All Users/Anwendungsdaten/BOINC/projects/climateprediction.net/txf/Palatino-BoldItalic.txf CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3412, selfPID=3412, iMonCtr=2 08:54:48 (900): No heartbeat from core client for 30 sec - exiting 08:55:35 (900): No heartbeat from core client for 30 sec - exiting 08:55:54 (900): No heartbeat from core client for 30 sec - exiting 08:55:55 (900): No heartbeat from core client for 30 sec - exiting 08:55:59 (900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:16:20 (5684): No heartbeat from core client for 30 sec - exiting 11:16:27 (5684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:20:09 (5120): No heartbeat from core client for 30 sec - exiting 11:21:52 (5120): No heartbeat from core client for 30 sec - exiting 11:21:53 (5120): No heartbeat from core client for 30 sec - exiting 11:21:54 (5120): No heartbeat from core client for 30 sec - exiting 11:21:55 (5120): No heartbeat from core client for 30 sec - exiting 11:21:56 (5120): No heartbeat from core client for 30 sec - exiting 11:21:57 (5120): No heartbeat from core client for 30 sec - exiting 11:21:58 (5120): No heartbeat from core client for 30 sec - exiting 11:21:59 (5120): No heartbeat from core client for 30 sec - exiting 11:22:00 (5120): No heartbeat from core client for 30 sec - exiting 11:22:01 (5120): No heartbeat from core client for 30 sec - exiting 11:22:02 (5120): No heartbeat from core client for 30 sec - exiting 11:22:03 (5120): No heartbeat from core client for 30 sec - exiting 11:22:04 (5120): No heartbeat from core client for 30 sec - exiting 11:22:05 (5120): No heartbeat from core client for 30 sec - exiting 11:22:06 (5120): No heartbeat from core client for 30 sec - exiting 11:22:07 (5120): No heartbeat from core client for 30 sec - exiting 11:22:08 (5120): No heartbeat from core client for 30 sec - exiting 11:22:09 (5120): No heartbeat from core client for 30 sec - exiting 11:22:10 (5120): No heartbeat from core client for 30 sec - exiting 11:22:11 (5120): No heartbeat from core client for 30 sec - exiting 11:22:12 (5120): No heartbeat from core client for 30 sec - exiting 11:22:13 (5120): No heartbeat from core client for 30 sec - exiting 11:22:14 (5120): No heartbeat from core client for 30 sec - exiting 11:22:15 (5120): No heartbeat from core client for 30 sec - exiting 11:22:16 (5120): No heartbeat from core client for 30 sec - exiting 11:22:17 (5120): No heartbeat from core client for 30 sec - exiting 11:22:18 (5120): No heartbeat from core client for 30 sec - exiting 11:22:19 (5120): No heartbeat from core client for 30 sec - exiting 11:22:20 (5120): No heartbeat from core client for 30 sec - exiting 11:22:22 (5120): No heartbeat from core client for 30 sec - exiting 11:22:23 (5120): No heartbeat from core client for 30 sec - exiting 11:22:24 (5120): No heartbeat from core client for 30 sec - exiting 11:22:25 (5120): No heartbeat from core client for 30 sec - exiting 11:22:26 (5120): No heartbeat from core client for 30 sec - exiting 11:22:27 (5120): No heartbeat from core client for 30 sec - exiting 11:22:28 (5120): No heartbeat from core client for 30 sec - exiting 11:22:29 (5120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:10 (5384): No heartbeat from core client for 30 sec - exiting 11:26:48 (5384): No heartbeat from core client for 30 sec - exiting 11:26:49 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:46:53 (5520): No heartbeat from core client for 30 sec - exiting 12:47:05 (5520): No heartbeat from core client for 30 sec - exiting 12:47:06 (5520): No heartbeat from core client for 30 sec - exiting 12:47:37 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:47:38 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 08:42:55 (2572): No heartbeat from core client for 30 sec - exiting 08:43:35 (2572): No heartbeat from core client for 30 sec - exiting 08:43:36 (2572): No heartbeat from core client for 30 sec - exiting 08:43:37 (2572): No heartbeat from core client for 30 sec - exiting 08:43:39 (2572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3052, selfPID=3052, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5120, selfPID=5120, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CP13:19:19 (4820): Can't set up shared mem: -1. Will run in standalone mode. 13:19:20 (6032): Can't set up shared mem: -1. Will run in standalone mode. No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6032, selfPID=4820, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6032, selfPID=6032, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4344, selfPID=4344, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5548, selfPID=5548, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6056, selfPID=6056, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5276, selfPID=5276, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=280, selfPID=280, iMonCtr=2 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4344, selfPID=4344, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5616, selfPID=5616, iMonCtr=2 CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2816, selfPID=2124, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2816, selfPID=2816, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:16:19 (6060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5848, selfPID=5848, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5336, selfPID=5336, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3872, selfPID=3872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bReCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1336, selfPID=1336, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4752, selfPID=4752, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=720, selfPID=720, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=568, selfPID=568, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3940, selfPID=3940, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1056, selfPID=1056, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3192, selfPID=3192, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3360, selfPID=3360, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4072, selfPID=4072, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:18:36 (1512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2112, selfPID=2112, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2984, selfPID=2984, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1176, selfPID=1176, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3052, selfPID=3052, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2264, selfPID=2264, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1692, selfPID=1692, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2884, selfPID=2884, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4072, selfPID=4072, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=4552, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4428, selfPID=4428, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4428, selfPID=3616, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 5 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_z4ol_1980_1_006926989_2_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_z4ol_1980_1_006926989_2_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_z4ol_1980_1_006926989_2_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_z4ol_1980_1_006926989_2_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_z4ol_1980_1_006926989_2_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_z4ol_1980_1_006926989_2_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_z4ol_1980_1_006926989_2_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 May 2011 08:27:06	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	57,696	333,170	5.7746
17 May 2011 14:58:59	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	46,176	270,434	5.8566
05 May 2011 13:18:35	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	34,662	210,396	6.0699
05 May 2011 13:03:20	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	34,660	209,722	6.0508
05 May 2011 12:22:35	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	34,656	209,091	6.0333
02 May 2011 10:36:03	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	23,136	144,790	6.2582
08 Apr 2011 10:40:42	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	11,646	77,581	6.6616
08 Apr 2011 08:59:48	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	11,641	76,658	6.5852
08 Apr 2011 07:48:35	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	11,631	75,791	6.5163
07 Apr 2011 14:07:46	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	11,625	75,001	6.4517
07 Apr 2011 12:19:22	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	11,618	74,178	6.3847
07 Apr 2011 11:35:49	1100062	12756099	hadam3p_pnw_z4ol_1980_1_006926989_2	11,616	73,394	6.3184