Task 12205444

Name	hadam3p_pnw_z3h3_1998_1_006925423_0
Workunit	7128739
Created	22 Nov 2010, 11:23:35 UTC
Sent	9 Apr 2011, 15:26:17 UTC
Report deadline	21 Mar 2012, 20:46:17 UTC
Received	17 May 2011, 21:26:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1017586
Run time	3 days 20 hours 58 min 3 sec
CPU time	3 days 4 hours 31 min 10 sec
Validate state	Invalid
Credit	1,754.30
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.10.36</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4884, selfPID=4884, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7088, selfPID=5688, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6652, selfPID=6652, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5220, selfPID=5220, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5432, selfPID=4568, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1460, selfPID=1632, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:55:13 (7368): Can't acquire lockfile (32) - waiting 35s 06:59:53 (8032): No heartbeat from core client for 30 sec - exiting 06:59:54 (8032): No heartbeat from core client for 30 sec - exiting 06:59:55 (8032): No heartbeat from core client for 30 sec - exiting 06:59:56 (8032): No heartbeat from core client for 30 sec - exiting 06:59:57 (8032): No heartbeat from core client for 30 sec - exiting 06:59:58 (8032): No heartbeat from core client for 30 sec - exiting 06:59:59 (8032): No heartbeat from core client for 30 sec - exiting 07:00:00 (8032): No heartbeat from core client for 30 sec - exiting 07:00:01 (8032): No heartbeat from core client for 30 sec - exiting 07:00:02 (8032): No heartbeat from core client for 30 sec - exiting 07:00:03 (8032): No heartbeat from core client for 30 sec - exiting 07:00:04 (8032): No heartbeat from core client for 30 sec - exiting 07:00:05 (8032): No heartbeat from core client for 30 sec - exiting 07:00:06 (8032): No heartbeat from core client for 30 sec - exiting 07:00:07 (8032): No heartbeat from core client for 30 sec - exiting 07:00:08 (8032): No heartbeat from core client for 30 sec - exiting 07:00:09 (8032): No heartbeat from core client for 30 sec - exiting 07:00:10 (8032): No heartbeat from core client for 30 sec - exiting 07:00:11 (8032): No heartbeat from core client for 30 sec - exiting 07:00:12 (8032): No heartbeat from core client for 30 sec - exiting 07:00:13 (8032): No heartbeat from core client for 30 sec - exiting 07:00:14 (8032): No heartbeat from core client for 30 sec - exiting 07:00:15 (8032): No heartbeat from core client for 30 sec - exiting 07:00:16 (8032): No heartbeat from core client for 30 sec - exiting 07:00:17 (8032): No heartbeat from core client for 30 sec - exiting 07:00:18 (8032): No heartbeat from core client for 30 sec - exiting 07:00:19 (8032): No heartbeat from core client for 30 sec - exiting 07:00:20 (8032): No heartbeat from core client for 30 sec - exiting 07:00:21 (8032): No heartbeat from core client for 30 sec - exiting 07:00:22 (8032): No heartbeat from core client for 30 sec - exiting 07:00:23 (8032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6432, selfPID=6432, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4468, selfPID=4468, iMonCtr=2 07:18:52 (6684): No heartbeat from core client for 30 sec - exiting 07:18:53 (6684): No heartbeat from core client for 30 sec - exiting 07:18:54 (6684): No heartbeat from core client for 30 sec - exiting 07:18:55 (6684): No heartbeat from core client for 30 sec - exiting 07:18:56 (6684): No heartbeat from core client for 30 sec - exiting 07:18:57 (6684): No heartbeat from core client for 30 sec - exiting 07:18:58 (6684): No heartbeat from core client for 30 sec - exiting 07:18:59 (6684): No heartbeat from core client for 30 sec - exiting 07:19:00 (6684): No heartbeat from core client for 30 sec - exiting 07:19:01 (6684): No heartbeat from core client for 30 sec - exiting 07:19:02 (6684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6848, selfPID=6848, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:20:39 (3932): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3240, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5136, iMonCtr=2 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 1 CPDN Monitor - Quit request from BOINC... 19:22:49 (5244): Can't acquire lockfile (32) - waiting 35s Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5244, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7068, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5856, selfPID=4888, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3996, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7856, selfPID=640, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2384, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7660, selfPID=7660, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5132, selfPID=5132, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:05:07 (5652): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:16:12 (6220): No heartbeat from core client for 30 sec - exiting 08:16:13 (6220): No heartbeat from core client for 30 sec - exiting 08:16:14 (6220): No heartbeat from core client for 30 sec - exiting 08:16:15 (6220): No heartbeat from core client for 30 sec - exiting 08:16:16 (6220): No heartbeat from core client for 30 sec - exiting 08:16:17 (6220): No heartbeat from core client for 30 sec - exiting 08:16:18 (6220): No heartbeat from core client for 30 sec - exiting 08:16:19 (6220): No heartbeat from core client for 30 sec - exiting 08:16:20 (6220): No heartbeat from core client for 30 sec - exiting 08:16:21 (6220): No heartbeat from core client for 30 sec - exiting 08:16:22 (6220): No heartbeat from core client for 30 sec - exiting 08:16:23 (6220): No heartbeat from core client for 30 sec - exiting 08:16:24 (6220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4392, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7104, iMonCtr=2 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6316, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7012, selfPID=7012, iMonCtr=2 18:43:52 (6248): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6364, selfPID=6364, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=716, selfPID=716, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8016, selfPID=8016, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:03:24 (656): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... 12:09:12 (6408): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:01:10 (8140): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8112, selfPID=8112, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=612, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:49:35 (4144): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... 17:40:25 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:12:18 (5140): No heartbeat from core client for 30 sec - exiting 09:12:19 (5140): No heartbeat from core client for 30 sec - exiting 09:12:20 (5140): No heartbeat from core client for 30 sec - exiting 09:12:21 (5140): No heartbeat from core client for 30 sec - exiting 09:12:22 (5140): No heartbeat from core client for 30 sec - exiting 09:12:23 (5140): No heartbeat from core client for 30 sec - exiting 09:12:24 (5140): No heartbeat from core client for 30 sec - exiting 09:12:25 (5140): No heartbeat from core client for 30 sec - exiting 09:12:26 (5140): No heartbeat from core client for 30 sec - exiting 09:12:27 (5140): No heartbeat from core client for 30 sec - exiting 09:12:28 (5140): No heartbeat from core client for 30 sec - exiting 09:12:29 (5140): No heartbeat from core client for 30 sec - exiting 09:12:30 (5140): No heartbeat from core client for 30 sec - exiting 09:12:31 (5140): No heartbeat from core client for 30 sec - exiting 09:12:32 (5140): No heartbeat from core client for 30 sec - exiting 09:12:33 (5140): No heartbeat from core client for 30 sec - exiting 09:12:34 (5140): No heartbeat from core client for 30 sec - exiting 09:12:35 (5140): No heartbeat from core client for 30 sec - exiting 09:12:36 (5140): No heartbeat from core client for 30 sec - exiting 09:12:37 (5140): No heartbeat from core client for 30 sec - exiting 09:12:38 (5140): No heartbeat from core client for 30 sec - exiting 09:12:39 (5140): No heartbeat from core client for 30 sec - exiting 09:12:40 (5140): No heartbeat from core client for 30 sec - exiting 09:12:41 (5140): No heartbeat from core client for 30 sec - exiting 09:12:42 (5140): No heartbeat from core client for 30 sec - exiting 09:12:43 (5140): No heartbeat from core client for 30 sec - exiting 09:12:44 (5140): No heartbeat from core client for 30 sec - exiting 09:12:45 (5140): No heartbeat from core client for 30 sec - exiting 09:12:46 (5140): No heartbeat from core client for 30 sec - exiting 09:12:47 (5140): No heartbeat from core client for 30 sec - exiting 09:12:48 (5140): No heartbeat from core client for 30 sec - exiting 09:12:49 (5140): No heartbeat from core client for 30 sec - exiting 09:12:50 (5140): No heartbeat from core client for 30 sec - exiting 09:12:51 (5140): No heartbeat from core client for 30 sec - exiting 09:12:52 (5140): No heartbeat from core client for 30 sec - exiting 09:12:53 (5140): No heartbeat from core client for 30 sec - exiting 09:12:54 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:47:40 (7344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:47:41 (7344): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7740, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7776, selfPID=4696, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 CPDN Monitor - Quit request from BOINC... 18:37:13 (7440): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5700, selfPID=3980, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2676, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6356, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5192, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7576, selfPID=7576, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4892, selfPID=2988, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5876, selfPID=6052, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:02:13 (6248): No heartbeat from core client for 30 sec - exiting 20:02:14 (6248): No heartbeat from core client for 30 sec - exiting 20:02:15 (6248): No heartbeat from core client for 30 sec - exiting 20:02:16 (6248): No heartbeat from core client for 30 sec - exiting 20:02:17 (6248): No heartbeat from core client for 30 sec - exiting 20:02:18 (6248): No heartbeat from core client for 30 sec - exiting 20:02:19 (6248): No heartbeat from core client for 30 sec - exiting 20:02:20 (6248): No heartbeat from core client for 30 sec - exiting 20:02:21 (6248): No heartbeat from core client for 30 sec - exiting 20:02:22 (6248): No heartbeat from core client for 30 sec - exiting 20:02:23 (6248): No heartbeat from core client for 30 sec - exiting 20:02:24 (6248): No heartbeat from core client for 30 sec - exiting 20:02:25 (6248): No heartbeat from core client for 30 sec - exiting 20:02:26 (6248): No heartbeat from core client for 30 sec - exiting 20:02:27 (6248): No heartbeat from core client for 30 sec - exiting 20:02:28 (6248): No heartbeat from core client for 30 sec - exiting 20:02:29 (6248): No heartbeat from core client for 30 sec - exiting 20:02:30 (6248): No heartbeat from core client for 30 sec - exiting 20:02:31 (6248): No heartbeat from core client for 30 sec - exiting 20:02:32 (6248): No heartbeat from core client for 30 sec - exiting 20:02:33 (6248): No heartbeat from core client for 30 sec - exiting 20:02:34 (6248): No heartbeat from core client for 30 sec - exiting 20:02:35 (6248): No heartbeat from core client for 30 sec - exiting 20:02:36 (6248): No heartbeat from core client for 30 sec - exiting 20:02:37 (6248): No heartbeat from core client for 30 sec - exiting 20:02:38 (6248): No heartbeat from core client for 30 sec - exiting 20:02:39 (6248): No heartbeat from core client for 30 sec - exiting 20:02:40 (6248): No heartbeat from core client for 30 sec - exiting 20:02:41 (6248): No heartbeat from core client for 30 sec - exiting 20:02:42 (6248): No heartbeat from core client for 30 sec - exiting 20:02:43 (6248): No heartbeat from core client for 30 sec - exiting 20:02:44 (6248): No heartbeat from core client for 30 sec - exiting 20:02:45 (6248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=604, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7708, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 May 2011 06:22:38	1017586	12205444	hadam3p_pnw_z3h3_1998_1_006925423_0	80,736	243,862	3.0205
03 May 2011 21:28:11	1017586	12205444	hadam3p_pnw_z3h3_1998_1_006925423_0	69,216	208,372	3.0105
29 Apr 2011 09:53:34	1017586	12205444	hadam3p_pnw_z3h3_1998_1_006925423_0	57,696	176,336	3.0563
26 Apr 2011 19:03:09	1017586	12205444	hadam3p_pnw_z3h3_1998_1_006925423_0	46,176	141,243	3.0588
22 Apr 2011 09:40:49	1017586	12205444	hadam3p_pnw_z3h3_1998_1_006925423_0	34,656	105,862	3.0547
20 Apr 2011 19:30:04	1017586	12205444	hadam3p_pnw_z3h3_1998_1_006925423_0	23,136	70,910	3.0649
12 Apr 2011 20:46:19	1017586	12205444	hadam3p_pnw_z3h3_1998_1_006925423_0	11,616	35,335	3.0419