Task 19229896

Name	hadam3p_anz_k16q_201212_12_306_010264683_1
Workunit	10264683
Created	28 Jan 2016, 20:29:29 UTC
Sent	28 Jan 2016, 20:35:53 UTC
Report deadline	10 Jan 2017, 1:55:53 UTC
Received	16 Mar 2016, 21:31:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1384825
Run time	3 days 5 hours 5 min 41 sec
CPU time	3 days 1 hours 30 min 15 sec
Validate state	Invalid
Credit	2,497.00
Device peak FLOPS	4.09 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <stderr_txt> diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 20:43:58 (9628): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 20:43:59 (4508): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:40:23 (11424): start_timer_thread(): CreateThread() failed, errno 0 16:40:25 (7332): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 11:18:46 (13620): start_timer_thread(): CreateThread() failed, errno 0 11:18:48 (7652): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 13:58:48 (9424): start_timer_thread(): CreateThread() failed, errno 0 13:58:49 (12344): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 19:11:11 (1116): start_timer_thread(): CreateThread() failed, errno 0 19:11:12 (4296): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:28:26 (8116): start_timer_thread(): CreateThread() failed, errno 0 10:28:27 (2928): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:09:28 (4520): start_timer_thread(): CreateThread() failed, errno 0 20:09:29 (6828): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 19:56:06 (6228): start_timer_thread(): CreateThread() failed, errno 0 19:56:08 (7672): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:32:22 (10864): start_timer_thread(): CreateThread() failed, errno 0 12:32:23 (7424): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:52:23 (4620): start_timer_thread(): CreateThread() failed, errno 0 11:52:24 (7760): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 17:14:29 (768): start_timer_thread(): CreateThread() failed, errno 0 17:14:31 (4928): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 18:23:21 (7040): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:59:39 (2916): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:59:40 (1976): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:42:14 (7680): start_timer_thread(): CreateThread() failed, errno 0 18:42:16 (9128): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:39:00 (1416): start_timer_thread(): CreateThread() failed, errno 0 10:39:00 (8168): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 19:44:01 (3736): start_timer_thread(): CreateThread() failed, errno 0 19:44:02 (6560): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:03:24 (10048): start_timer_thread(): CreateThread() failed, errno 0 22:03:26 (6004): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 16:58:44 (7660): No heartbeat from core client for 30 sec - exiting 16:58:45 (7660): No heartbeat from core client for 30 sec - exiting 16:58:46 (7660): No heartbeat from core client for 30 sec - exiting 16:58:47 (7660): No heartbeat from core client for 30 sec - exiting 16:58:48 (7660): No heartbeat from core client for 30 sec - exiting 16:58:49 (7660): No heartbeat from core client for 30 sec - exiting 16:58:50 (7660): No heartbeat from core client for 30 sec - exiting 16:58:51 (7660): No heartbeat from core client for 30 sec - exiting 16:58:52 (7660): No heartbeat from core client for 30 sec - exiting 16:58:53 (7660): No heartbeat from core client for 30 sec - exiting 16:58:54 (7660): No heartbeat from core client for 30 sec - exiting 16:58:55 (7660): No heartbeat from core client for 30 sec - exiting 16:58:56 (7660): No heartbeat from core client for 30 sec - exiting 16:58:57 (7660): No heartbeat from core client for 30 sec - exiting 16:59:20 (856): start_timer_thread(): CreateThread() failed, errno 0 16:59:21 (5456): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - No 'heartbeat' from BOINC... 16:59:30 (12760): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 19:34:13 (5460): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 19:34:16 (7020): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:31:23 (2868): start_timer_thread(): CreateThread() failed, errno 0 10:31:25 (4868): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 10:32:23 (5956): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 10:32:24 (8804): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:24:49 (4808): start_timer_thread(): CreateThread() failed, errno 0 20:24:51 (11076): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 08:14:09 (6448): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 08:14:11 (2972): start_timer_thread(): CreateThread() failed, errno 0 08:14:51 (7136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 08:15:28 (8432): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 08:15:30 (8884): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:48:46 (7128): start_timer_thread(): CreateThread() failed, errno 0 10:48:47 (4580): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:24:30 (7528): start_timer_thread(): CreateThread() failed, errno 0 18:24:31 (7592): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:41:54 (152): start_timer_thread(): CreateThread() failed, errno 0 09:41:56 (11700): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:09:27 (7748): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:09:28 (5340): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 14:49:44 (5632): start_timer_thread(): CreateThread() failed, errno 0 14:49:46 (7072): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:21:33 (5636): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:21:35 (11536): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:19:28 (14908): start_timer_thread(): CreateThread() failed, errno 0 20:19:29 (7048): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 21:57:48 (13332): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 21:57:50 (10940): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 20:08:59 (2904): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 20:09:00 (15164): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 21:25:01 (12820): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 21:25:03 (10624): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:56:30 (9124): start_timer_thread(): CreateThread() failed, errno 0 10:56:32 (3856): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:59:07 (9612): start_timer_thread(): CreateThread() failed, errno 0 09:59:08 (5328): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:17:15 (12104): start_timer_thread(): CreateThread() failed, errno 0 09:17:16 (6372): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:11:16 (4776): start_timer_thread(): CreateThread() failed, errno 0 11:11:18 (3464): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:31:50 (3960): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:42:28 (10296): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 20:02:40 (1240): start_timer_thread(): CreateThread() failed, errno 0 20:02:40 (9124): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 21:36:27 (6004): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 19:23:26 (4436): start_timer_thread(): CreateThread() failed, errno 0 19:23:27 (6840): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:22:55 (4772): start_timer_thread(): CreateThread() failed, errno 0 13:23:03 (11296): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=4772, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11296, selfPID=12096, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_k16q_201212_12_306_010264683_1_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k16q_201212_12_306_010264683_1_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k16q_201212_12_306_010264683_1_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k16q_201212_12_306_010264683_1_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k16q_201212_12_306_010264683_1_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k16q_201212_12_306_010264683_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k16q_201212_12_306_010264683_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Mar 2016 21:32:21	1384825	19229896	hadam3p_anz_k16q_201212_12_306_010264683_1	57,899	251,308	4.3405
05 Mar 2016 20:49:02	1384825	19229896	hadam3p_anz_k16q_201212_12_306_010264683_1	46,379	201,199	4.3381
29 Feb 2016 13:49:05	1384825	19229896	hadam3p_anz_k16q_201212_12_306_010264683_1	34,859	151,617	4.3494
27 Feb 2016 16:48:20	1384825	19229896	hadam3p_anz_k16q_201212_12_306_010264683_1	23,339	101,184	4.3354
08 Feb 2016 21:03:49	1384825	19229896	hadam3p_anz_k16q_201212_12_306_010264683_1	11,819	50,906	4.3071