Task 17580386

Name	hadam3p_anz_e2v8_2012_1_009294217_0
Workunit	9378405
Created	17 Dec 2014, 18:10:59 UTC
Sent	10 Jan 2015, 21:47:08 UTC
Report deadline	24 Dec 2015, 3:07:08 UTC
Received	21 Feb 2015, 12:45:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1351518
Run time	1 days 23 hours 58 min 2 sec
CPU time	1 days 22 hours 45 min 54 sec
Validate state	Invalid
Credit	1,006.54
Device peak FLOPS	3.40 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:21:28 (5396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7024, selfPID=4364, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 18:18:29 (5360): No heartbeat from core client for 30 sec - exiting 18:18:30 (5360): No heartbeat from core client for 30 sec - exiting 18:18:31 (5360): No heartbeat from core client for 30 sec - exiting 18:18:32 (5360): No heartbeat from core client for 30 sec - exiting 18:18:33 (5360): No heartbeat from core client for 30 sec - exiting 18:18:34 (5360): No heartbeat from core client for 30 sec - exiting 18:18:35 (5360): No heartbeat from core client for 30 sec - exiting 18:18:36 (5360): No heartbeat from core client for 30 sec - exiting 18:18:37 (5360): No heartbeat from core client for 30 sec - exiting 18:18:38 (5360): No heartbeat from core client for 30 sec - exiting 18:18:39 (5360): No heartbeat from core client for 30 sec - exiting 18:18:40 (5360): No heartbeat from core client for 30 sec - exiting 18:18:41 (5360): No heartbeat from core client for 30 sec - exiting 18:18:42 (5360): No heartbeat from core client for 30 sec - exiting 18:18:43 (5360): No heartbeat from core client for 30 sec - exiting 18:18:44 (5360): No heartbeat from core client for 30 sec - exiting 18:18:45 (5360): No heartbeat from core client for 30 sec - exiting 18:18:46 (5360): No heartbeat from core client for 30 sec - exiting 18:18:47 (5360): No heartbeat from core client for 30 sec - exiting 18:18:48 (5360): No heartbeat from core client for 30 sec - exiting 18:18:49 (5360): No heartbeat from core client for 30 sec - exiting 18:18:50 (5360): No heartbeat from core client for 30 sec - exiting 18:18:52 (5360): No heartbeat from core client for 30 sec - exiting 18:18:53 (5360): No heartbeat from core client for 30 sec - exiting 18:18:54 (5360): No heartbeat from core client for 30 sec - exiting 18:18:55 (5360): No heartbeat from core client for 30 sec - exiting 18:18:56 (5360): No heartbeat from core client for 30 sec - exiting 18:18:57 (5360): No heartbeat from core client for 30 sec - exiting 18:18:58 (5360): No heartbeat from core client for 30 sec - exiting 18:18:59 (5360): No heartbeat from core client for 30 sec - exiting 18:19:00 (5360): No heartbeat from core client for 30 sec - exiting 18:19:01 (5360): No heartbeat from core client for 30 sec - exiting 18:19:02 (5360): No heartbeat from core client for 30 sec - exiting 18:19:03 (5360): No heartbeat from core client for 30 sec - exiting 18:19:04 (5360): No heartbeat from core client for 30 sec - exiting 18:19:05 (5360): No heartbeat from core client for 30 sec - exiting 18:19:06 (5360): No heartbeat from core client for 30 sec - exiting 18:19:07 (5360): No heartbeat from core client for 30 sec - exiting 18:19:08 (5360): No heartbeat from core client for 30 sec - exiting 18:19:09 (5360): No heartbeat from core client for 30 sec - exiting 18:19:10 (5360): No heartbeat from core client for 30 sec - exiting 18:19:11 (5360): No heartbeat from core client for 30 sec - exiting 18:19:12 (5360): No heartbeat from core client for 30 sec - exiting 18:19:13 (5360): No heartbeat from core client for 30 sec - exiting 18:19:14 (5360): No heartbeat from core client for 30 sec - exiting 18:19:15 (5360): No heartbeat from core client for 30 sec - exiting 18:19:16 (5360): No heartbeat from core client for 30 sec - exiting 18:19:17 (5360): No heartbeat from core client for 30 sec - exiting 18:19:18 (5360): No heartbeat from core client for 30 sec - exiting 18:19:19 (5360): No heartbeat from core client for 30 sec - exiting 18:19:20 (5360): No heartbeat from core client for 30 sec - exiting 18:19:21 (5360): No heartbeat from core client for 30 sec - exiting 18:19:22 (5360): No heartbeat from core client for 30 sec - exiting 18:19:23 (5360): No heartbeat from core client for 30 sec - exiting 18:19:24 (5360): No heartbeat from core client for 30 sec - exiting 18:19:25 (5360): No heartbeat from core client for 30 sec - exiting 18:19:26 (5360): No heartbeat from core client for 30 sec - exiting 18:19:27 (5360): No heartbeat from core client for 30 sec - exiting 18:19:28 (5360): No heartbeat from core client for 30 sec - exiting 18:19:29 (5360): No heartbeat from core client for 30 sec - exiting 18:19:30 (5360): No heartbeat from core client for 30 sec - exiting 18:19:31 (5360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:35:51 (4228): No heartbeat from core client for 30 sec - exiting 18:35:52 (4228): No heartbeat from core client for 30 sec - exiting 18:35:53 (4228): No heartbeat from core client for 30 sec - exiting 18:35:54 (4228): No heartbeat from core client for 30 sec - exiting 18:35:55 (4228): No heartbeat from core client for 30 sec - exiting 18:35:56 (4228): No heartbeat from core client for 30 sec - exiting 18:35:57 (4228): No heartbeat from core client for 30 sec - exiting 18:35:58 (4228): No heartbeat from core client for 30 sec - exiting 18:35:59 (4228): No heartbeat from core client for 30 sec - exiting 18:36:00 (4228): No heartbeat from core client for 30 sec - exiting 18:36:01 (4228): No heartbeat from core client for 30 sec - exiting 18:36:02 (4228): No heartbeat from core client for 30 sec - exiting 18:36:03 (4228): No heartbeat from core client for 30 sec - exiting 18:36:04 (4228): No heartbeat from core client for 30 sec - exiting 18:36:05 (4228): No heartbeat from core client for 30 sec - exiting 18:36:06 (4228): No heartbeat from core client for 30 sec - exiting 18:36:07 (4228): No heartbeat from core client for 30 sec - exiting 18:36:08 (4228): No heartbeat from core client for 30 sec - exiting 18:36:09 (4228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:10:32 (5856): No heartbeat from core client for 30 sec - exiting 23:10:33 (5856): No heartbeat from core client for 30 sec - exiting 23:10:34 (5856): No heartbeat from core client for 30 sec - exiting 23:10:35 (5856): No heartbeat from core client for 30 sec - exiting 23:10:36 (5856): No heartbeat from core client for 30 sec - exiting 23:10:37 (5856): No heartbeat from core client for 30 sec - exiting 23:10:38 (5856): No heartbeat from core client for 30 sec - exiting 23:10:39 (5856): No heartbeat from core client for 30 sec - exiting 23:10:40 (5856): No heartbeat from core client for 30 sec - exiting 23:10:41 (5856): No heartbeat from core client for 30 sec - exiting 23:10:42 (5856): No heartbeat from core client for 30 sec - exiting 23:10:43 (5856): No heartbeat from core client for 30 sec - exiting 23:10:44 (5856): No heartbeat from core client for 30 sec - exiting 23:10:45 (5856): No heartbeat from core client for 30 sec - exiting 23:10:46 (5856): No heartbeat from core client for 30 sec - exiting 23:10:47 (5856): No heartbeat from core client for 30 sec - exiting 23:10:48 (5856): No heartbeat from core client for 30 sec - exiting 23:10:49 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2176, selfPID=2176, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5112, selfPID=5340, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2v8_2012_1_009294217_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Feb 2015 15:53:29	1351518	17580386	hadam3p_anz_e2v8_2012_1_009294217_0	23,339	116,756	5.0026
12 Jan 2015 03:31:27	1351518	17580386	hadam3p_anz_e2v8_2012_1_009294217_0	11,819	58,946	4.9874