Task 19198782

Name	hadam3p_anz_q3gq_201212_12_290_010260989_0
Workunit	10260989
Created	14 Jan 2016, 15:40:22 UTC
Sent	14 Jan 2016, 20:22:56 UTC
Report deadline	27 Dec 2016, 1:42:56 UTC
Received	27 Feb 2016, 14:41:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1374093
Run time	16 hours 52 min 37 sec
CPU time	16 hours 17 min 58 sec
Validate state	Invalid
Credit	1,006.54
Device peak FLOPS	4.02 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.9</core_client_version> <![CDATA[ <stderr_txt> 21:33:02 (5492): start_timer_thread(): CreateThread() failed, errno 0 21:33:04 (2628): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:03:20 (1224): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:27:27 (5156): start_timer_thread(): CreateThread() failed, errno 0 22:27:28 (10484): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:49:35 (7536): start_timer_thread(): CreateThread() failed, errno 0 22:49:37 (2496): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 22:23:09 (12264): start_timer_thread(): CreateThread() failed, errno 0 22:23:11 (7936): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:14:47 (5116): start_timer_thread(): CreateThread() failed, errno 0 23:14:48 (3664): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:26:27 (1224): start_timer_thread(): CreateThread() failed, errno 0 23:26:29 (9524): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:51:53 (6388): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:41:58 (11060): start_timer_thread(): CreateThread() failed, errno 0 18:41:59 (10884): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:54:55 (2688): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:41:56 (11780): start_timer_thread(): CreateThread() failed, errno 0 21:41:58 (11252): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:50:46 (5264): start_timer_thread(): CreateThread() failed, errno 0 22:50:48 (5344): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:57:56 (10260): start_timer_thread(): CreateThread() failed, errno 0 22:57:58 (5096): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:07:11 (11224): start_timer_thread(): CreateThread() failed, errno 0 23:07:12 (5468): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:14:46 (11584): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 21:28:36 (9688): start_timer_thread(): CreateThread() failed, errno 0 21:28:38 (6372): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:53:55 (232): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:50:44 (5780): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:54:55 (6736): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:34:32 (2012): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:16:24 (7748): start_timer_thread(): CreateThread() failed, errno 0 21:16:26 (12040): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:06:39 (12836): start_timer_thread(): CreateThread() failed, errno 0 23:06:41 (2636): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 22:30:19 (5172): start_timer_thread(): CreateThread() failed, errno 0 22:30:21 (8244): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... 21:04:47 (6388): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:51:14 (6764): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:28:58 (10324): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:38:24 (12900): start_timer_thread(): CreateThread() failed, errno 0 23:38:26 (3360): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 00:25:24 (7848): start_timer_thread(): CreateThread() failed, errno 0 00:25:26 (7720): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:59:24 (11056): start_timer_thread(): CreateThread() failed, errno 0 00:59:26 (2792): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 20:33:38 (9544): start_timer_thread(): CreateThread() failed, errno 0 20:33:40 (11356): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 22:30:25 (3076): start_timer_thread(): CreateThread() failed, errno 0 22:30:26 (12636): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:50:17 (7504): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:04:15 (12304): start_timer_thread(): CreateThread() failed, errno 0 19:04:17 (6092): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:01:36 (4252): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:22:07 (12088): start_timer_thread(): CreateThread() failed, errno 0 19:22:08 (13044): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:43:46 (10620): start_timer_thread(): CreateThread() failed, errno 0 20:43:48 (6676): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:59:02 (3256): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:05:36 (7684): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:35:11 (8320): start_timer_thread(): CreateThread() failed, errno 0 23:35:13 (8124): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:06:31 (10664): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 00:10:19 (8356): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:23:54 (92): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 01:27:03 (11352): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:42:58 (9452): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:34:53 (7292): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:49:27 (8404): start_timer_thread(): CreateThread() failed, errno 0 14:40:37 (7096): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 14:40:38 (1760): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1760, selfPID=9460, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_q3gq_201212_12_290_010260989_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Feb 2016 20:29:37	1374093	19198782	hadam3p_anz_q3gq_201212_12_290_010260989_0	23,339	53,891	2.3091
06 Feb 2016 05:44:10	1374093	19198782	hadam3p_anz_q3gq_201212_12_290_010260989_0	11,819	27,235	2.3043