Task 17589702

Name	hadam3p_anz_l3md_2012_1_009303476_0
Workunit	9387664
Created	17 Dec 2014, 19:24:54 UTC
Sent	24 Dec 2014, 8:39:05 UTC
Report deadline	6 Dec 2015, 13:59:05 UTC
Received	29 Sep 2015, 16:30:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1350570
Run time	1 days 17 hours 18 min 7 sec
CPU time	1 days 16 hours 24 min 12 sec
Validate state	Invalid
Credit	1,006.54
Device peak FLOPS	2.97 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.27</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:35:18 (8116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:24 (8116): No heartbeat from core client for 30 sec - exiting 17:35:25 (8116): No heartbeat from core client for 30 sec - exiting 17:35:27 (8116): No heartbeat from core client for 30 sec - exiting 17:35:28 (8116): No heartbeat from core client for 30 sec - exiting 17:35:29 (8116): No heartbeat from core client for 30 sec - exiting 17:35:30 (8116): No heartbeat from core client for 30 sec - exiting 17:35:31 (8116): No heartbeat from core client for 30 sec - exiting 17:35:33 (8116): No heartbeat from core client for 30 sec - exiting 17:35:34 (8116): No heartbeat from core client for 30 sec - exiting 17:35:35 (8116): No heartbeat from core client for 30 sec - exiting 17:35:36 (8116): No heartbeat from core client for 30 sec - exiting 17:35:37 (8116): No heartbeat from core client for 30 sec - exiting 17:35:39 (8116): No heartbeat from core client for 30 sec - exiting 17:35:40 (8116): No heartbeat from core client for 30 sec - exiting 17:35:41 (8116): No heartbeat from core client for 30 sec - exiting 17:35:42 (8116): No heartbeat from core client for 30 sec - exiting 17:35:43 (8116): No heartbeat from core client for 30 sec - exiting 17:35:44 (8116): No heartbeat from core client for 30 sec - exiting 17:35:46 (8116): No heartbeat from core client for 30 sec - exiting 17:35:47 (8116): No heartbeat from core client for 30 sec - exiting 17:35:48 (8116): No heartbeat from core client for 30 sec - exiting 17:35:49 (8116): No heartbeat from core client for 30 sec - exiting 17:35:50 (8116): No heartbeat from core client for 30 sec - exiting 17:35:51 (8116): No heartbeat from core client for 30 sec - exiting 17:35:52 (8116): No heartbeat from core client for 30 sec - exiting 17:35:53 (8116): No heartbeat from core client for 30 sec - exiting 17:35:55 (8116): No heartbeat from core client for 30 sec - exiting 17:35:56 (8116): No heartbeat from core client for 30 sec - exiting 17:35:57 (8116): No heartbeat from core client for 30 sec - exiting 17:35:58 (8116): No heartbeat from core client for 30 sec - exiting 17:35:59 (8116): No heartbeat from core client for 30 sec - exiting 17:36:00 (8116): No heartbeat from core client for 30 sec - exiting 17:36:01 (8116): No heartbeat from core client for 30 sec - exiting 17:36:02 (8116): No heartbeat from core client for 30 sec - exiting 17:36:04 (8116): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6376, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 08:10:28 (6608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:34:50 (6428): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:34:51 (6428): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:16:03 (6760): No heartbeat from core client for 30 sec - exiting 09:16:04 (6760): No heartbeat from core client for 30 sec - exiting 09:16:05 (6760): No heartbeat from core client for 30 sec - exiting 09:16:06 (6760): No heartbeat from core client for 30 sec - exiting 09:16:07 (6760): No heartbeat from core client for 30 sec - exiting 09:16:08 (6760): No heartbeat from core client for 30 sec - exiting 09:16:09 (6760): No heartbeat from core client for 30 sec - exiting 09:16:10 (6760): No heartbeat from core client for 30 sec - exiting 09:16:11 (6760): No heartbeat from core client for 30 sec - exiting 09:16:12 (6760): No heartbeat from core client for 30 sec - exiting 09:16:13 (6760): No heartbeat from core client for 30 sec - exiting 09:16:14 (6760): No heartbeat from core client for 30 sec - exiting 09:16:15 (6760): No heartbeat from core client for 30 sec - exiting 09:16:16 (6760): No heartbeat from core client for 30 sec - exiting 09:16:17 (6760): No heartbeat from core client for 30 sec - exiting 09:16:18 (6760): No heartbeat from core client for 30 sec - exiting 09:16:19 (6760): No heartbeat from core client for 30 sec - exiting 09:16:20 (6760): No heartbeat from core client for 30 sec - exiting 09:16:21 (6760): No heartbeat from core client for 30 sec - exiting 09:16:22 (6760): No heartbeat from core client for 30 sec - exiting 09:16:23 (6760): No heartbeat from core client for 30 sec - exiting 09:16:24 (6760): No heartbeat from core client for 30 sec - exiting 09:16:25 (6760): No heartbeat from core client for 30 sec - exiting 09:16:26 (6760): No heartbeat from core client for 30 sec - exiting 09:16:27 (6760): No heartbeat from core client for 30 sec - exiting 09:16:28 (6760): No heartbeat from core client for 30 sec - exiting 09:16:29 (6760): No heartbeat from core client for 30 sec - exiting 09:16:30 (6760): No heartbeat from core client for 30 sec - exiting 09:16:31 (6760): No heartbeat from core client for 30 sec - exiting 09:16:32 (6760): No heartbeat from core client for 30 sec - exiting 09:16:33 (6760): No heartbeat from core client for 30 sec - exiting 09:16:34 (6760): No heartbeat from core client for 30 sec - exiting 09:16:35 (6760): No heartbeat from core client for 30 sec - exiting 09:16:36 (6760): No heartbeat from core client for 30 sec - exiting 09:16:37 (6760): No heartbeat from core client for 30 sec - exiting 09:16:38 (6760): No heartbeat from core client for 30 sec - exiting 09:16:39 (6760): No heartbeat from core client for 30 sec - exiting 09:16:40 (6760): No heartbeat from core client for 30 sec - exiting 09:16:41 (6760): No heartbeat from core client for 30 sec - exiting 09:16:42 (6760): No heartbeat from core client for 30 sec - exiting 09:16:43 (6760): No heartbeat from core client for 30 sec - exiting 09:16:44 (6760): No heartbeat from core client for 30 sec - exiting 09:16:45 (6760): No heartbeat from core client for 30 sec - exiting 09:16:46 (6760): No heartbeat from core client for 30 sec - exiting 09:16:47 (6760): No heartbeat from core client for 30 sec - exiting 09:16:48 (6760): No heartbeat from core client for 30 sec - exiting 09:16:49 (6760): No heartbeat from core client for 30 sec - exiting 09:16:50 (6760): No heartbeat from core client for 30 sec - exiting 09:16:51 (6760): No heartbeat from core client for 30 sec - exiting 09:16:52 (6760): No heartbeat from core client for 30 sec - exiting 09:16:53 (6760): No heartbeat from core client for 30 sec - exiting 09:16:54 (6760): No heartbeat from core client for 30 sec - exiting 09:16:55 (6760): No heartbeat from core client for 30 sec - exiting 09:16:56 (6760): No heartbeat from core client for 30 sec - exiting 09:16:57 (6760): No heartbeat from core client for 30 sec - exiting 09:16:58 (6760): No heartbeat from core client for 30 sec - exiting 09:16:59 (6760): No heartbeat from core client for 30 sec - exiting 09:17:00 (6760): No heartbeat from core client for 30 sec - exiting 09:17:01 (6760): No heartbeat from core client for 30 sec - exiting 09:17:02 (6760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:36:28 (11640): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:07:33 (7060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 22 WARNING: BOINC Windows Runtime Debugger has been disabled. Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x753A812F BOINC Windows Runtime Debugger not configured, terminating application... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10520, selfPID=11640, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_l3md_2012_1_009303476_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Sep 2015 19:09:13	1350570	17589702	hadam3p_anz_l3md_2012_1_009303476_0	23,339	131,100	5.6172
18 Sep 2015 15:31:48	1350570	17589702	hadam3p_anz_l3md_2012_1_009303476_0	11,819	61,818	5.2304