Task 18353766

Name	hadam3p_anz_d1k6_2013_1_009721169_1
Workunit	9794466
Created	25 Apr 2015, 13:59:01 UTC
Sent	25 Apr 2015, 14:07:34 UTC
Report deadline	6 Apr 2016, 19:27:34 UTC
Received	5 May 2015, 2:35:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1333395
Run time	5 days 16 hours 9 min 14 sec
CPU time	5 days 10 hours 58 min 58 sec
Validate state	Invalid
Credit	4,981.10
Device peak FLOPS	3.83 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <stderr_txt> 00:12:19 (18148): start_timer_thread(): CreateThread() failed, errno 0 00:12:21 (16560): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:05:56 (12668): start_timer_thread(): CreateThread() failed, errno 0 01:05:58 (5392): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 04:04:41 (9460): start_timer_thread(): CreateThread() failed, errno 0 04:04:43 (15300): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9460, selfPID=9460, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9460, selfPID=15300, iMonCtr=1 11:03:59 (4584): start_timer_thread(): CreateThread() failed, errno 0 11:04:01 (9236): start_timer_thread(): CreateThread() failed, errno 0 13:07:58 (15048): start_timer_thread(): CreateThread() failed, errno 0 13:08:00 (12220): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15048, selfPID=15048, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15048, selfPID=12220, iMonCtr=1 11:03:22 (8336): start_timer_thread(): CreateThread() failed, errno 0 11:03:24 (10244): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:13:43 (10824): start_timer_thread(): CreateThread() failed, errno 0 18:13:45 (17388): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10824, selfPID=10824, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10824, selfPID=17388, iMonCtr=1 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 19:12:29 (6868): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 19:12:31 (15112): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6868, selfPID=6868, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6868, selfPID=15112, iMonCtr=1 00:55:46 (3020): start_timer_thread(): CreateThread() failed, errno 0 00:55:47 (13284): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 03:04:02 (17780): start_timer_thread(): CreateThread() failed, errno 0 03:04:03 (8104): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 08:56:16 (16380): start_timer_thread(): CreateThread() failed, errno 0 08:56:17 (1060): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:26:58 (18332): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 13:49:37 (16724): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 13:49:39 (15172): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:14:25 (14688): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 14:14:27 (6048): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:07:09 (9108): start_timer_thread(): CreateThread() failed, errno 0 19:07:11 (15572): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9108, selfPID=9108, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9108, selfPID=15572, iMonCtr=1 23:45:18 (9020): start_timer_thread(): CreateThread() failed, errno 0 23:45:20 (16916): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 10:45:25 (18336): start_timer_thread(): CreateThread() failed, errno 0 10:45:27 (15180): start_timer_thread(): CreateThread() failed, errno 0 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18336, selfPID=18336, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18336, selfPID=15180, iMonCtr=1 11:27:59 (14480): start_timer_thread(): CreateThread() failed, errno 0 11:28:02 (12756): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14480, selfPID=12756, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14480, selfPID=14480, iMonCtr=1 15:45:31 (4920): start_timer_thread(): CreateThread() failed, errno 0 15:45:33 (17712): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:43:05 (12748): start_timer_thread(): CreateThread() failed, errno 0 12:43:07 (4616): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:57:20 (4988): start_timer_thread(): CreateThread() failed, errno 0 14:00:35 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:00:55 (13896): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:07:49 (15324): start_timer_thread(): CreateThread() failed, errno 0 13:07:51 (9252): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:45:51 (22656): start_timer_thread(): CreateThread() failed, errno 0 00:45:52 (28380): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:50:47 (4512): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 01:50:49 (11072): start_timer_thread(): CreateThread() failed, errno 0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11072, selfPID=8912, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_d1k6_2013_1_009721169_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_d1k6_2013_1_009721169_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 May 2015 19:11:33	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	115,499	448,301	3.8814
08 May 2015 19:10:18	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	103,979	407,576	3.9198
08 May 2015 19:07:40	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	92,459	366,606	3.9651
08 May 2015 19:06:17	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	80,939	325,498	4.0215
08 May 2015 19:03:00	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	69,419	284,371	4.0964
30 Apr 2015 02:51:59	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	57,899	238,427	4.1180
29 Apr 2015 13:36:37	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	46,379	191,214	4.1229
28 Apr 2015 23:36:26	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	34,859	143,633	4.1204
27 Apr 2015 21:47:55	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	23,339	96,068	4.1162
27 Apr 2015 02:55:46	1333395	18353766	hadam3p_anz_d1k6_2013_1_009721169_1	11,819	48,585	4.1108