Task 19199260

Name	hadam3p_anz_q3to_201212_12_290_010261455_0
Workunit	10261455
Created	14 Jan 2016, 15:48:42 UTC
Sent	15 Jan 2016, 2:50:30 UTC
Report deadline	27 Dec 2016, 8:10:30 UTC
Received	2 Apr 2016, 5:28:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1386986
Run time	2 days 17 hours 3 min 51 sec
CPU time	2 days 16 hours 48 min 3 sec
Validate state	Invalid
Credit	2,000.18
Device peak FLOPS	2.94 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <stderr_txt> 17:07:04 (7372): start_timer_thread(): CreateThread() failed, errno 0 17:07:06 (3948): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:59:54 (27948): start_timer_thread(): CreateThread() failed, errno 0 02:59:56 (23068): start_timer_thread(): CreateThread() failed, errno 0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=31488, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 16:00:13 (14300): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 10:13:55 (13772): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 02:06:47 (9248): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 08:34:31 (8148): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:00:01 (23108): start_timer_thread(): CreateThread() failed, errno 0 16:00:03 (7584): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:17:56 (22956): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:44:01 (25836): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:00:02 (28404): start_timer_thread(): CreateThread() failed, errno 0 16:00:03 (15840): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 01:00:10 (3120): start_timer_thread(): CreateThread() failed, errno 0 01:00:11 (13792): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 17:25:49 (19532): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 17:25:50 (26228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:00:02 (25320): start_timer_thread(): CreateThread() failed, errno 0 16:00:03 (25552): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:00:02 (27060): start_timer_thread(): CreateThread() failed, errno 0 16:00:03 (40040): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:44:54 (11440): start_timer_thread(): CreateThread() failed, errno 0 19:44:56 (38384): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 03:29:53 (41520): start_timer_thread(): CreateThread() failed, errno 0 03:29:55 (40996): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:00:09 (18240): start_timer_thread(): CreateThread() failed, errno 0 16:00:10 (9984): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:53:48 (23244): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:53:50 (5524): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:41:31 (12096): start_timer_thread(): CreateThread() failed, errno 0 02:41:33 (10864): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 08:06:47 (8904): start_timer_thread(): CreateThread() failed, errno 0 08:06:49 (5840): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 16:00:02 (24124): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 16:00:04 (14604): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 00:22:08 (38384): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 00:22:09 (7772): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 05:37:57 (23768): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 05:37:58 (17172): start_timer_thread(): CreateThread() failed, errno 0 06:43:39 (3268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:43:40 (3268): No heartbeat from core client for 30 sec - exiting 06:43:41 (3268): No heartbeat from core client for 30 sec - exiting 06:43:42 (3268): No heartbeat from core client for 30 sec - exiting 06:43:43 (3268): No heartbeat from core client for 30 sec - exiting 06:43:44 (3268): No heartbeat from core client for 30 sec - exiting 06:43:45 (3268): No heartbeat from core client for 30 sec - exiting 06:43:46 (3268): No heartbeat from core client for 30 sec - exiting 06:43:47 (3268): No heartbeat from core client for 30 sec - exiting 06:43:48 (3268): No heartbeat from core client for 30 sec - exiting 06:43:49 (3268): No heartbeat from core client for 30 sec - exiting 06:43:50 (3268): No heartbeat from core client for 30 sec - exiting 06:43:51 (3268): No heartbeat from core client for 30 sec - exiting 06:43:52 (3268): No heartbeat from core client for 30 sec - exiting 06:43:53 (3268): No heartbeat from core client for 30 sec - exiting 06:43:54 (3268): No heartbeat from core client for 30 sec - exiting diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 06:43:57 (24912): start_timer_thread(): CreateThread() failed, errno 0 06:43:58 (264): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 22:02:26 (26748): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 22:02:28 (23108): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 16:19:10 (27912): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 16:19:12 (4616): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 19:15:42 (17628): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 19:15:44 (14028): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 06:24:06 (24964): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 06:24:07 (31264): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:43:04 (11820): start_timer_thread(): CreateThread() failed, errno 0 19:43:05 (12668): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:07:19 (4304): start_timer_thread(): CreateThread() failed, errno 0 20:07:20 (13920): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:10:50 (18788): start_timer_thread(): CreateThread() failed, errno 0 20:10:51 (18420): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:14:38 (13992): start_timer_thread(): CreateThread() failed, errno 0 20:14:39 (3188): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:18:55 (10444): start_timer_thread(): CreateThread() failed, errno 0 20:18:56 (16624): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:23:19 (16564): start_timer_thread(): CreateThread() failed, errno 0 20:23:20 (18368): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:27:30 (11548): start_timer_thread(): CreateThread() failed, errno 0 20:27:31 (12296): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:59:39 (16552): start_timer_thread(): CreateThread() failed, errno 0 20:59:40 (1128): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:03:23 (19340): start_timer_thread(): CreateThread() failed, errno 0 21:03:25 (15916): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:35:25 (13316): start_timer_thread(): CreateThread() failed, errno 0 21:35:26 (10784): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:39:24 (5236): start_timer_thread(): CreateThread() failed, errno 0 21:39:25 (9728): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:44:44 (7448): start_timer_thread(): CreateThread() failed, errno 0 21:44:45 (18228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:49:23 (9128): start_timer_thread(): CreateThread() failed, errno 0 21:49:24 (10988): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:53:23 (10000): start_timer_thread(): CreateThread() failed, errno 0 21:53:24 (2596): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:59:45 (7904): start_timer_thread(): CreateThread() failed, errno 0 21:59:46 (6552): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:04:45 (15752): start_timer_thread(): CreateThread() failed, errno 0 22:04:46 (11880): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:09:24 (19152): start_timer_thread(): CreateThread() failed, errno 0 22:09:25 (11468): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:13:24 (15556): start_timer_thread(): CreateThread() failed, errno 0 22:13:25 (7272): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:17:10 (17108): start_timer_thread(): CreateThread() failed, errno 0 22:17:11 (10792): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:20:50 (11328): start_timer_thread(): CreateThread() failed, errno 0 22:20:51 (18496): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:24:47 (17364): start_timer_thread(): CreateThread() failed, errno 0 22:24:48 (10820): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:29:26 (11844): start_timer_thread(): CreateThread() failed, errno 0 22:29:27 (10872): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:33:25 (17080): start_timer_thread(): CreateThread() failed, errno 0 22:33:26 (16100): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:39:49 (15992): start_timer_thread(): CreateThread() failed, errno 0 22:39:50 (720): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:44:48 (3348): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:49:28 (14296): start_timer_thread(): CreateThread() failed, errno 0 22:49:29 (16112): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:53:27 (20124): start_timer_thread(): CreateThread() failed, errno 0 22:53:28 (20428): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:59:50 (19492): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:04:49 (20212): start_timer_thread(): CreateThread() failed, errno 0 23:04:50 (6216): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:09:30 (20280): start_timer_thread(): CreateThread() failed, errno 0 23:09:31 (21096): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:13:27 (20668): start_timer_thread(): CreateThread() failed, errno 0 23:13:28 (18932): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:17:47 (21112): start_timer_thread(): CreateThread() failed, errno 0 23:17:48 (20816): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:22:17 (8280): start_timer_thread(): CreateThread() failed, errno 0 23:22:18 (13780): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:27:54 (21056): start_timer_thread(): CreateThread() failed, errno 0 23:27:56 (7248): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:31:51 (17468): start_timer_thread(): CreateThread() failed, errno 0 23:31:53 (10436): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:36:12 (6036): start_timer_thread(): CreateThread() failed, errno 0 23:36:13 (18252): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:39:54 (21144): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:44:51 (4532): start_timer_thread(): CreateThread() failed, errno 0 23:44:52 (20912): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:48:27 (18292): start_timer_thread(): CreateThread() failed, errno 0 23:48:28 (13360): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:52:24 (15944): start_timer_thread(): CreateThread() failed, errno 0 23:52:26 (6128): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:56:13 (18004): start_timer_thread(): CreateThread() failed, errno 0 23:56:14 (12248): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:59:55 (20364): start_timer_thread(): CreateThread() failed, errno 0 23:59:56 (16388): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20364, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16388, selfPID=4984, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3to_201212_12_290_010261455_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Mar 2016 16:39:14	1386986	19199260	hadam3p_anz_q3to_201212_12_290_010261455_0	46,379	191,238	4.1234
09 Mar 2016 15:17:05	1386986	19199260	hadam3p_anz_q3to_201212_12_290_010261455_0	34,859	143,230	4.1088
12 Feb 2016 09:59:53	1386986	19199260	hadam3p_anz_q3to_201212_12_290_010261455_0	23,339	96,531	4.1360
11 Feb 2016 12:33:04	1386986	19199260	hadam3p_anz_q3to_201212_12_290_010261455_0	11,819	48,484	4.1022