Task 19231682

Name	hadam3p_anz_n236_201412_12_315_010280320_0
Workunit	10280320
Created	29 Jan 2016, 10:49:05 UTC
Sent	30 Jan 2016, 10:27:04 UTC
Report deadline	11 Jan 2017, 15:47:04 UTC
Received	1 Feb 2016, 14:58:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1371876
Run time	1 days 0 hours 59 min 14 sec
CPU time	1 days 0 hours 2 min 39 sec
Validate state	Invalid
Credit	1,006.54
Device peak FLOPS	4.14 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <stderr_txt> diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 10:42:56 (1640): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 10:42:58 (1212): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING:diOagnostics_init_unhandledgger htion_monitor()edCr eating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime): Creaer hhas been disabled. o 0 11:34:02 (16036): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15616, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:25:30 (11776): No heartbeat from core client for 30 sec - exiting 07:25:31 (11776): No heartbeat from core client for 30 sec - exiting 07:25:32 (11776): No heartbeat from core client for 30 sec - exiting 07:25:34 (11776): No heartbeat from core client for 30 sec - exiting 07:25:35 (11776): No heartbeat from core client for 30 sec - exiting 07:25:36 (11776): No heartbeat from core client for 30 sec - exiting 07:25:37 (11776): No heartbeat from core client for 30 sec - exiting 07:25:38 (11776): No heartbeat from core client for 30 sec - exiting 07:25:39 (11776): No heartbeat from core client for 30 sec - exiting 07:25:40 (11776): No heartbeat from core client for 30 sec - exiting 07:25:41 (11776): No heartbeat from core client for 30 sec - exiting 07:25:42 (11776): No heartbeat from core client for 30 sec - exiting 07:25:43 (11776): No heartbeat from core client for 30 sec - exiting 07:25:44 (11776): No heartbeat from core client for 30 sec - exiting 07:25:45 (11776): No heartbeat from core client for 30 sec - exiting 07:25:46 (11776): No heartbeat from core client for 30 sec - exiting 07:25:47 (11776): No heartbeat from core client for 30 sec - exiting 07:25:48 (11776): No heartbeat from core client for 30 sec - exiting 07:25:50 (11776): No heartbeat from core client for 30 sec - exiting 07:25:51 (11776): No heartbeat from core client for 30 sec - exiting 07:25:52 (11776): No heartbeat from core client for 30 sec - exiting 07:25:53 (11776): No heartbeat from core client for 30 sec - exiting 07:25:54 (11776): No heartbeat from core client for 30 sec - exiting 07:25:55 (11776): No heartbeat from core client for 30 sec - exiting 07:25:56 (11776): No heartbeat from core client for 30 sec - exiting 07:25:57 (11776): No heartbeat from core client for 30 sec - exiting 07:25:58 (11776): No heartbeat from core client for 30 sec - exiting 07:25:59 (11776): No heartbeat from core client for 30 sec - exiting 07:26:00 (11776): No heartbeat from core client for 30 sec - exiting 07:26:01 (11776): No heartbeat from core client for 30 sec - exiting 07:26:03 (11776): No heartbeat from core client for 30 sec - exiting 07:26:04 (11776): No heartbeat from core client for 30 sec - exiting 07:26:05 (11776): No heartbeat from core client for 30 sec - exiting 07:26:06 (11776): No heartbeat from core client for 30 sec - exiting 07:26:07 (11776): No heartbeat from core client for 30 sec - exiting 07:26:08 (11776): No heartbeat from core client for 30 sec - exiting 07:26:09 (11776): No heartbeat from core client for 30 sec - exiting 07:26:10 (11776): No heartbeat from core client for 30 sec - exiting 07:26:11 (11776): No heartbeat from core client for 30 sec - exiting 07:26:12 (11776): No heartbeat from core client for 30 sec - exiting 07:26:13 (11776): No heartbeat from core client for 30 sec - exiting 07:26:14 (11776): No heartbeat from core client for 30 sec - exiting 07:26:15 (11776): No heartbeat from core client for 30 sec - exiting 07:26:16 (11776): No heartbeat from core client for 30 sec - exiting 07:26:17 (11776): No heartbeat from core client for 30 sec - exiting 07:26:19 (11776): No heartbeat from core client for 30 sec - exiting 07:26:20 (11776): No heartbeat from core client for 30 sec - exiting 07:26:21 (11776): No heartbeat from core client for 30 sec - exiting 07:26:22 (11776): No heartbeat from core client for 30 sec - exiting 07:26:23 (11776): No heartbeat from core client for 30 sec - exiting 07:26:24 (11776): No heartbeat from core client for 30 sec - exiting 07:26:25 (11776): No heartbeat from core client for 30 sec - exiting 07:26:26 (11776): No heartbeat from core client for 30 sec - exiting 07:26:27 (11776): No heartbeat from core client for 30 sec - exiting 07:26:28 (11776): No heartbeat from core client for 30 sec - exiting 07:26:29 (11776): No heartbeat from core client for 30 sec - exiting 07:26:30 (11776): No heartbeat from core client for 30 sec - exiting 07:26:31 (11776): No heartbeat from core client for 30 sec - exiting 07:26:32 (11776): No heartbeat from core client for 30 sec - exiting 07:26:34 (11776): No heartbeat from core client for 30 sec - exiting 07:26:35 (11776): No heartbeat from core client for 30 sec - exiting 07:26:36 (11776): No heartbeat from core client for 30 sec - exiting 07:26:37 (11776): No heartbeat from core client for 30 sec - exiting 07:26:38 (11776): No heartbeat from core client for 30 sec - exiting 07:26:39 (11776): No heartbeat from core client for 30 sec - exiting 07:26:40 (11776): No heartbeat from core client for 30 sec - exiting 07:26:41 (11776): No heartbeat from core client for 30 sec - exiting 07:26:42 (11776): No heartbeat from core client for 30 sec - exiting 07:26:43 (11776): No heartbeat from core client for 30 sec - exiting 07:26:44 (11776): No heartbeat from core client for 30 sec - exiting 07:26:45 (11776): No heartbeat from core client for 30 sec - exiting 07:26:46 (11776): No heartbeat from core client for 30 sec - exiting 07:26:47 (11776): No heartbeat from core client for 30 sec - exiting 07:26:49 (11776): No heartbeat from core client for 30 sec - exiting 07:26:50 (11776): No heartbeat from core client for 30 sec - exiting 07:26:51 (11776): No heartbeat from core client for 30 sec - exiting diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:26:52 (12904): start_timer_thread(): CreateThread() failed, errno 0 07:26:52 (11776): No heartbeat from core client for 30 sec - exiting 07:26:53 (11776): No heartbeat from core client for 30 sec - exiting diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:26:53 (12964): start_timer_thread(): CreateThread() failed, errno 0 07:26:54 (11776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:27:07 (13160): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:27:09 (4432): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13160, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4432, selfPID=12924, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 07:45:29 (10036): No heartbeat from core client for 30 sec - exiting 07:45:30 (10036): No heartbeat from core client for 30 sec - exiting 07:45:31 (10036): No heartbeat from core client for 30 sec - exiting 07:45:32 (10036): No heartbeat from core client for 30 sec - exiting 07:45:33 (10036): No heartbeat from core client for 30 sec - exiting 07:45:34 (10036): No heartbeat from core client for 30 sec - exiting 07:45:35 (10036): No heartbeat from core client for 30 sec - exiting 07:45:36 (10036): No heartbeat from core client for 30 sec - exiting 07:45:37 (10036): No heartbeat from core client for 30 sec - exiting 07:45:38 (10036): No heartbeat from core client for 30 sec - exiting 07:45:39 (10036): No heartbeat from core client for 30 sec - exiting 07:45:40 (10036): No heartbeat from core client for 30 sec - exiting 07:45:41 (10036): No heartbeat from core client for 30 sec - exiting 07:45:43 (10036): No heartbeat from core client for 30 sec - exiting 07:45:44 (10036): No heartbeat from core client for 30 sec - exiting 07:45:45 (10036): No heartbeat from core client for 30 sec - exiting 07:45:46 (10036): No heartbeat from core client for 30 sec - exiting 07:45:47 (10036): No heartbeat from core client for 30 sec - exiting 07:45:48 (10036): No heartbeat from core client for 30 sec - exiting 07:45:49 (10036): No heartbeat from core client for 30 sec - exiting 07:45:50 (10036): No heartbeat from core client for 30 sec - exiting 07:45:51 (10036): No heartbeat from core client for 30 sec - exiting 07:45:52 (10036): No heartbeat from core client for 30 sec - exiting 07:45:53 (10036): No heartbeat from core client for 30 sec - exiting 07:45:54 (10036): No heartbeat from core client for 30 sec - exiting 07:45:55 (10036): No heartbeat from core client for 30 sec - exiting 07:45:57 (10036): No heartbeat from core client for 30 sec - exiting 07:45:58 (10036): No heartbeat from core client for 30 sec - exiting 07:45:59 (10036): No heartbeat from core client for 30 sec - exiting 07:46:00 (10036): No heartbeat from core client for 30 sec - exiting 07:46:01 (10036): No heartbeat from core client for 30 sec - exiting 07:46:02 (10036): No heartbeat from core client for 30 sec - exiting 07:46:03 (10036): No heartbeat from core client for 30 sec - exiting 07:46:04 (10036): No heartbeat from core client for 30 sec - exiting 07:46:05 (10036): No heartbeat from core client for 30 sec - exiting 07:46:06 (10036): No heartbeat from core client for 30 sec - exiting 07:46:07 (10036): No heartbeat from core client for 30 sec - exiting 07:46:08 (10036): No heartbeat from core client for 30 sec - exiting 07:46:09 (10036): No heartbeat from core client for 30 sec - exiting 07:46:11 (10036): No heartbeat from core client for 30 sec - exiting 07:46:12 (10036): No heartbeat from core client for 30 sec - exiting 07:46:13 (10036): No heartbeat from core client for 30 sec - exiting 07:46:14 (10036): No heartbeat from core client for 30 sec - exiting 07:46:15 (10036): No heartbeat from core client for 30 sec - exiting 07:46:16 (10036): No heartbeat from core client for 30 sec - exiting 07:46:17 (10036): No heartbeat from core client for 30 sec - exiting 07:46:18 (10036): No heartbeat from core client for 30 sec - exiting 07:46:19 (10036): No heartbeat from core client for 30 sec - exiting 07:46:20 (10036): No heartbeat from core client for 30 sec - exiting 07:46:21 (10036): No heartbeat from core client for 30 sec - exiting 07:46:22 (10036): No heartbeat from core client for 30 sec - exiting 07:46:23 (11868): start_timer_thread(): CreateThread() failed, errno 0 07:46:23 (10036): No heartbeat from core client for 30 sec - exiting 07:46:24 (8004): start_timer_thread(): CreateThread() failed, errno 0 07:46:25 (10036): No heartbeat from core client for 30 sec - exiting 07:46:26 (10036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:29 (2612): start_timer_thread(): CreateThread() failed, errno 0 07:46:31 (12304): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:21:16 (15336): start_timer_thread(): CreateThread() failed, errno 0 13:21:18 (15504): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:46:59 (2364): start_timer_thread(): CreateThread() failed, errno 0 18:47:04 (7260): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:41:21 (15652): No heartbeat from core client for 30 sec - exiting 08:41:22 (15652): No heartbeat from core client for 30 sec - exiting 08:41:23 (15652): No heartbeat from core client for 30 sec - exiting 08:41:24 (15652): No heartbeat from core client for 30 sec - exiting 08:41:25 (15652): No heartbeat from core client for 30 sec - exiting 08:41:26 (15652): No heartbeat from core client for 30 sec - exiting 08:41:27 (15652): No heartbeat from core client for 30 sec - exiting 08:41:28 (15652): No heartbeat from core client for 30 sec - exiting 08:41:29 (15652): No heartbeat from core client for 30 sec - exiting 08:41:30 (15652): No heartbeat from core client for 30 sec - exiting 08:41:31 (15652): No heartbeat from core client for 30 sec - exiting 08:41:32 (15652): No heartbeat from core client for 30 sec - exiting 08:41:33 (15652): No heartbeat from core client for 30 sec - exiting 08:41:34 (15652): No heartbeat from core client for 30 sec - exiting 08:41:35 (15652): No heartbeat from core client for 30 sec - exiting 08:41:36 (15652): No heartbeat from core client for 30 sec - exiting 08:41:37 (15652): No heartbeat from core client for 30 sec - exiting 08:41:38 (15652): No heartbeat from core client for 30 sec - exiting 08:41:39 (15652): No heartbeat from core client for 30 sec - exiting 08:41:40 (15652): No heartbeat from core client for 30 sec - exiting 08:41:41 (15652): No heartbeat from core client for 30 sec - exiting 08:41:42 (15652): No heartbeat from core client for 30 sec - exiting 08:41:44 (15652): No heartbeat from core client for 30 sec - exiting 08:41:45 (15652): No heartbeat from core client for 30 sec - exiting 08:41:46 (15652): No heartbeat from core client for 30 sec - exiting 08:41:47 (15652): No heartbeat from core client for 30 sec - exiting 08:41:48 (15652): No heartbeat from core client for 30 sec - exiting 08:41:49 (15652): No heartbeat from core client for 30 sec - exiting 08:41:50 (15652): No heartbeat from core client for 30 sec - exiting 08:41:51 (15652): No heartbeat from core client for 30 sec - exiting 08:41:52 (15652): No heartbeat from core client for 30 sec - exiting 08:41:53 (15652): No heartbeat from core client for 30 sec - exiting 08:41:54 (15652): No heartbeat from core client for 30 sec - exiting 08:41:55 (15652): No heartbeat from core client for 30 sec - exiting 08:41:56 (15652): No heartbeat from core client for 30 sec - exiting 08:41:57 (15652): No heartbeat from core client for 30 sec - exiting 08:41:58 (15652): No heartbeat from core client for 30 sec - exiting 08:41:59 (15652): No heartbeat from core client for 30 sec - exiting 08:42:00 (15652): No heartbeat from core client for 30 sec - exiting 08:42:01 (15652): No heartbeat from core client for 30 sec - exiting 08:42:02 (15652): No heartbeat from core client for 30 sec - exiting 08:42:03 (15652): No heartbeat from core client for 30 sec - exiting 08:42:04 (15652): No heartbeat from core client for 30 sec - exiting 08:42:05 (15652): No heartbeat from core client for 30 sec - exiting 08:42:06 (15652): No heartbeat from core client for 30 sec - exiting 08:42:07 (15652): No heartbeat from core client for 30 sec - exiting 08:42:08 (15652): No heartbeat from core client for 30 sec - exiting 08:42:09 (15652): No heartbeat from core client for 30 sec - exiting 08:42:10 (15652): No heartbeat from core client for 30 sec - exiting 08:42:11 (15652): No heartbeat from core client for 30 sec - exiting 08:42:12 (15652): No heartbeat from core client for 30 sec - exiting 08:42:13 (15652): No heartbeat from core client for 30 sec - exiting 08:42:14 (15652): No heartbeat from core client for 30 sec - exiting 08:42:15 (15652): No heartbeat from core client for 30 sec - exiting 08:42:16 (15652): No heartbeat from core client for 30 sec - exiting 08:42:17 (15652): No heartbeat from core client for 30 sec - exiting 08:42:18 (15652): No heartbeat from core client for 30 sec - exiting 08:42:19 (15652): No heartbeat from core client for 30 sec - exiting 08:42:21 (15652): No heartbeat from core client for 30 sec - exiting 08:42:22 (15652): No heartbeat from core client for 30 sec - exiting 08:42:23 (15652): No heartbeat from core client for 30 sec - exiting 08:42:24 (15652): No heartbeat from core client for 30 sec - exiting 08:42:25 (15652): No heartbeat from core client for 30 sec - exiting 08:42:26 (15652): No heartbeat from core client for 30 sec - exiting 08:42:27 (15652): No heartbeat from core client for 30 sec - exiting 08:42:28 (15652): No heartbeat from core client for 30 sec - exiting 08:42:29 (15652): No heartbeat from core client for 30 sec - exiting 08:42:30 (15652): No heartbeat from core client for 30 sec - exiting 08:42:31 (15652): No heartbeat from core client for 30 sec - exiting 08:42:32 (15652): No heartbeat from core client for 30 sec - exiting 08:42:33 (15652): No heartbeat from core client for 30 sec - exiting 08:42:34 (15652): No heartbeat from core client for 30 sec - exiting 08:42:35 (15652): No heartbeat from core client for 30 sec - exiting 08:42:36 (15652): No heartbeat from core client for 30 sec - exiting 08:42:37 (18036): start_timer_thread(): CreateThread() failed, errno 0 08:42:37 (15652): No heartbeat from core client for 30 sec - exiting 08:42:38 (15652): No heartbeat from core client for 30 sec - exiting 08:42:38 (5036): start_timer_thread(): CreateThread() failed, errno 0 08:42:39 (15652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:42 (18272): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:41:37 (1244): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=1244, iMonCtr=1 14:41:44 (19904): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=19904, selfPID=15616, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n236_201412_12_315_010280320_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Jan 2016 08:21:54	1371876	19231682	hadam3p_anz_n236_201412_12_315_010280320_0	23,339	71,851	3.0786
30 Jan 2016 22:39:24	1371876	19231682	hadam3p_anz_n236_201412_12_315_010280320_0	11,819	36,429	3.0822