Task 19198192

Name	hadam3p_anz_q30o_201212_12_290_010260411_0
Workunit	10260411
Created	14 Jan 2016, 15:30:00 UTC
Sent	14 Jan 2016, 15:32:45 UTC
Report deadline	26 Dec 2016, 20:52:45 UTC
Received	15 Feb 2016, 13:47:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1365647
Run time	4 days 19 hours 3 min 57 sec
CPU time	4 days 4 hours 30 min 30 sec
Validate state	Invalid
Credit	3,987.46
Device peak FLOPS	3.10 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <stderr_txt> 16:33:09 (7568): start_timer_thread(): CreateThread() failed, errno 0 16:33:11 (7052): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:59:02 (3060): start_timer_thread(): CreateThread() failed, errno 0 11:59:03 (244): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:45:48 (2408): start_timer_thread(): CreateThread() failed, errno 0 02:45:50 (6520): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:58:05 (8872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:13:40 (9804): start_timer_thread(): CreateThread() failed, errno 0 12:13:42 (1236): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:07:24 (11420): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 07:22:55 (2092): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 20:08:51 (10356): start_timer_thread(): CreateThread() failed, errno 0 20:08:53 (10524): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:23:45 (4560): No heartbeat from core client for 30 sec - exiting 00:23:46 (4560): No heartbeat from core client for 30 sec - exiting 00:23:47 (4560): No heartbeat from core client for 30 sec - exiting 00:23:48 (4560): No heartbeat from core client for 30 sec - exiting 00:23:49 (4560): No heartbeat from core client for 30 sec - exiting 00:23:50 (4560): No heartbeat from core client for 30 sec - exiting 00:23:51 (4560): No heartbeat from core client for 30 sec - exiting 00:23:52 (4560): No heartbeat from core client for 30 sec - exiting 00:23:53 (4560): No heartbeat from core client for 30 sec - exiting 00:23:54 (3136): start_timer_thread(): CreateThread() failed, errno 0 00:23:54 (4560): No heartbeat from core client for 30 sec - exiting 00:23:56 (4560): No heartbeat from core client for 30 sec - exiting 00:23:56 (7632): start_timer_thread(): CreateThread() failed, errno 0 00:23:57 (4560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:24:31 (2536): start_timer_thread(): CreateThread() failed, errno 0 00:24:32 (4552): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 08:06:02 (5548): start_timer_thread(): CreateThread() failed, errno 0 08:06:04 (11224): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 06:39:08 (4348): start_timer_thread(): CreateThread() failed, errno 0 06:39:10 (11160): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:50:56 (8140): start_timer_thread(): CreateThread() failed, errno 0 08:50:57 (8420): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 11:21:18 (10668): start_timer_thread(): CreateThread() failed, errno 0 11:21:20 (11844): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:22:28 (10916): start_timer_thread(): CreateThread() failed, errno 0 19:22:29 (12612): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:31:07 (9484): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:25:14 (9144): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:08:43 (1588): start_timer_thread(): CreateThread() failed, errno 0 14:08:44 (1156): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:22:47 (9944): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:58:09 (8544): start_timer_thread(): CreateThread() failed, errno 0 16:58:10 (9444): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:04:10 (8272): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 08:14:54 (9204): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:47:34 (8144): start_timer_thread(): CreateThread() failed, errno 0 15:47:35 (11216): start_timer_thread(): CreateThread() failed, errno 0 15:47:39 (756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:47:41 (10228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:12:40 (5604): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:21:04 (3280): start_timer_thread(): CreateThread() failed, errno 0 19:21:06 (6780): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:01:38 (5260): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:57:27 (2840): start_timer_thread(): CreateThread() failed, errno 0 08:57:28 (7900): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:42:37 (1988): start_timer_thread(): CreateThread() failed, errno 0 17:42:39 (12192): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:34:06 (12680): start_timer_thread(): CreateThread() failed, errno 0 18:34:08 (7204): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 07:39:45 (11860): start_timer_thread(): CreateThread() failed, errno 0 07:39:47 (7228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:31:34 (12960): start_timer_thread(): CreateThread() failed, errno 0 15:31:35 (7228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:08:29 (6716): start_timer_thread(): CreateThread() failed, errno 0 22:08:30 (4476): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:34:34 (3308): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 11:05:36 (3584): start_timer_thread(): CreateThread() failed, errno 0 11:05:36 (11404): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 11:45:15 (9308): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:27:48 (10676): start_timer_thread(): CreateThread() failed, errno 0 12:27:50 (5848): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:35:23 (6920): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:02:02 (9876): start_timer_thread(): CreateThread() failed, errno 0 12:02:04 (7768): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:39:27 (2348): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:15:22 (7400): start_timer_thread(): CreateThread() failed, errno 0 15:15:23 (9848): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:16:55 (7644): start_timer_thread(): CreateThread() failed, errno 0 16:16:57 (3412): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:04:42 (4900): start_timer_thread(): CreateThread() failed, errno 0 18:04:43 (2792): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:57:34 (9756): start_timer_thread(): CreateThread() failed, errno 0 20:57:36 (9696): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:42:27 (8040): start_timer_thread(): CreateThread() failed, errno 0 06:42:28 (3992): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:58:03 (9800): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:46:05 (5048): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:46:08 (5016): start_timer_thread(): CreateThread() failed, errno 0 13:46:10 (10744): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:23:14 (8468): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 02:32:53 (9040): start_timer_thread(): CreateThread() failed, errno 0 02:32:55 (5824): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 04:18:15 (6872): start_timer_thread(): CreateThread() failed, errno 0 04:18:16 (8184): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:22:23 (4364): start_timer_thread(): CreateThread() failed, errno 0 06:22:25 (5964): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 09:24:57 (6044): start_timer_thread(): CreateThread() failed, errno 0 09:24:58 (4240): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:42:20 (10636): start_timer_thread(): CreateThread() failed, errno 0 14:42:21 (2488): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:47:03 (7484): start_timer_thread(): CreateThread() failed, errno 0 15:47:05 (11324): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:11:56 (5388): start_timer_thread(): CreateThread() failed, errno 0 16:11:57 (668): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:13:13 (6708): start_timer_thread(): CreateThread() failed, errno 0 17:13:14 (2796): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:05:04 (2492): start_timer_thread(): CreateThread() failed, errno 0 19:05:05 (2884): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:18:32 (4392): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 05:58:15 (4500): start_timer_thread(): CreateThread() failed, errno 0 05:58:16 (5272): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:50:00 (496): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 07:24:59 (4524): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 10:15:58 (2288): start_timer_thread(): CreateThread() failed, errno 0 10:15:59 (7880): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 11:14:30 (608): start_timer_thread(): CreateThread() failed, errno 0 11:14:32 (6716): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:20:25 (8576): start_timer_thread(): CreateThread() failed, errno 0 12:20:27 (3864): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:17:57 (3708): start_timer_thread(): CreateThread() failed, errno 0 18:17:59 (8648): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:51:11 (3548): start_timer_thread(): CreateThread() failed, errno 0 20:51:12 (9248): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 01:02:25 (632): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:50:39 (6256): start_timer_thread(): CreateThread() failed, errno 0 13:50:40 (9228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:25:11 (6932): start_timer_thread(): CreateThread() failed, errno 0 16:25:12 (2284): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:09:32 (5096): start_timer_thread(): CreateThread() failed, errno 0 19:09:33 (2056): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:51:52 (252): start_timer_thread(): CreateThread() failed, errno 0 20:51:54 (224): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 21:06:40 (5248): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 21:06:42 (6168): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:21:54 (1360): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:54:58 (768): start_timer_thread(): CreateThread() failed, errno 0 15:54:59 (8456): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:24:52 (5728): start_timer_thread(): CreateThread() failed, errno 0 18:24:54 (6136): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:59:32 (6868): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:51:01 (9092): start_timer_thread(): CreateThread() failed, errno 0 21:51:02 (8040): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:55:45 (7772): start_timer_thread(): CreateThread() failed, errno 0 06:55:46 (8636): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 07:11:18 (11448): start_timer_thread(): CreateThread() failed, errno 0 07:11:20 (8836): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:59:19 (5248): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:59:21 (8140): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 13:33:35 (9712): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 13:33:37 (10760): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:12:37 (8504): start_timer_thread(): CreateThread() failed, errno 0 14:12:39 (4824): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:26:15 (7520): start_timer_thread(): CreateThread() failed, errno 0 15:26:16 (4356): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:50:05 (6280): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 08:39:29 (1128): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 11:17:17 (6340): start_timer_thread(): CreateThread() failed, errno 0 11:17:19 (2424): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:45:54 (2852): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=2852, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5704, selfPID=5704, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5704, selfPID=6676, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_q30o_201212_12_290_010260411_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q30o_201212_12_290_010260411_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q30o_201212_12_290_010260411_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q30o_201212_12_290_010260411_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Feb 2016 18:06:50	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	92,459	338,072	3.6565
08 Feb 2016 10:34:43	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	80,939	296,085	3.6581
03 Feb 2016 17:55:10	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	69,419	254,493	3.6660
01 Feb 2016 09:01:14	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	57,899	213,009	3.6790
27 Jan 2016 19:58:06	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	46,379	170,753	3.6817
24 Jan 2016 12:49:45	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	34,859	129,154	3.7050
21 Jan 2016 10:41:40	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	23,339	87,031	3.7290
17 Jan 2016 14:06:03	1365647	19198192	hadam3p_anz_q30o_201212_12_290_010260411_0	11,819	44,053	3.7273