Task 18902981

Name	hadam3p_anz_w02p_2000_1_010175333_0
Workunit	10129489
Created	12 Sep 2015, 19:54:01 UTC
Sent	12 Sep 2015, 21:46:37 UTC
Report deadline	25 Aug 2016, 3:06:37 UTC
Received	17 Sep 2015, 19:06:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1318460
Run time	2 days 9 hours 9 min 6 sec
CPU time	2 days 0 hours 16 min 56 sec
Validate state	Invalid
Credit	2,497.00
Device peak FLOPS	3.88 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:04:24 (8148): No heartbeat from core client for 30 sec - exiting 18:04:25 (8148): No heartbeat from core client for 30 sec - exiting 18:04:26 (8148): No heartbeat from core client for 30 sec - exiting 18:04:27 (8148): No heartbeat from core client for 30 sec - exiting 18:04:28 (8148): No heartbeat from core client for 30 sec - exiting 18:04:29 (8148): No heartbeat from core client for 30 sec - exiting 18:04:30 (8148): No heartbeat from core client for 30 sec - exiting 18:04:31 (8148): No heartbeat from core client for 30 sec - exiting 18:04:32 (8148): No heartbeat from core client for 30 sec - exiting 18:04:33 (8148): No heartbeat from core client for 30 sec - exiting 18:04:34 (8148): No heartbeat from core client for 30 sec - exiting 18:04:35 (8148): No heartbeat from core client for 30 sec - exiting 18:04:36 (8148): No heartbeat from core client for 30 sec - exiting 18:04:37 (8148): No heartbeat from core client for 30 sec - exiting 18:04:38 (8148): No heartbeat from core client for 30 sec - exiting 18:04:39 (8148): No heartbeat from core client for 30 sec - exiting 18:04:40 (8148): No heartbeat from core client for 30 sec - exiting 18:04:41 (8148): No heartbeat from core client for 30 sec - exiting 18:04:42 (8148): No heartbeat from core client for 30 sec - exiting 18:04:43 (8148): No heartbeat from core client for 30 sec - exiting 18:04:44 (8148): No heartbeat from core client for 30 sec - exiting 18:04:45 (8148): No heartbeat from core client for 30 sec - exiting 18:04:46 (8148): No heartbeat from core client for 30 sec - exiting 18:04:47 (8148): No heartbeat from core client for 30 sec - exiting 18:04:48 (8148): No heartbeat from core client for 30 sec - exiting 18:04:49 (8148): No heartbeat from core client for 30 sec - exiting 18:04:50 (8148): No heartbeat from core client for 30 sec - exiting 18:04:51 (8148): No heartbeat from core client for 30 sec - exiting 18:04:52 (8148): No heartbeat from core client for 30 sec - exiting 18:04:53 (8148): No heartbeat from core client for 30 sec - exiting 18:04:54 (8148): No heartbeat from core client for 30 sec - exiting 18:04:55 (8148): No heartbeat from core client for 30 sec - exiting 18:04:56 (8148): No heartbeat from core client for 30 sec - exiting 18:04:57 (8148): No heartbeat from core client for 30 sec - exiting 18:04:58 (8148): No heartbeat from core client for 30 sec - exiting 18:04:59 (8148): No heartbeat from core client for 30 sec - exiting 18:05:00 (8148): No heartbeat from core client for 30 sec - exiting 18:05:01 (8148): No heartbeat from core client for 30 sec - exiting 18:05:02 (8148): No heartbeat from core client for 30 sec - exiting 18:05:03 (8148): No heartbeat from core client for 30 sec - exiting 18:05:04 (8148): No heartbeat from core client for 30 sec - exiting 18:05:05 (8148): No heartbeat from core client for 30 sec - exiting 18:05:06 (8148): No heartbeat from core client for 30 sec - exiting 18:05:07 (8148): No heartbeat from core client for 30 sec - exiting 18:05:08 (8148): No heartbeat from core client for 30 sec - exiting 18:05:09 (8148): No heartbeat from core client for 30 sec - exiting 18:05:10 (8148): No heartbeat from core client for 30 sec - exiting 18:05:11 (8148): No heartbeat from core client for 30 sec - exiting 18:05:12 (8148): No heartbeat from core client for 30 sec - exiting 18:05:13 (8148): No heartbeat from core client for 30 sec - exiting 18:05:14 (8148): No heartbeat from core client for 30 sec - exiting 18:05:15 (8148): No heartbeat from core client for 30 sec - exiting 18:05:16 (8148): No heartbeat from core client for 30 sec - exiting 18:05:17 (8148): No heartbeat from core client for 30 sec - exiting 18:05:18 (8148): No heartbeat from core client for 30 sec - exiting 18:05:19 (8148): No heartbeat from core client for 30 sec - exiting 18:05:20 (8148): No heartbeat from core client for 30 sec - exiting 18:05:21 (8148): No heartbeat from core client for 30 sec - exiting 18:05:22 (8148): No heartbeat from core client for 30 sec - exiting 18:05:23 (8148): No heartbeat from core client for 30 sec - exiting 18:05:24 (8148): No heartbeat from core client for 30 sec - exiting 18:05:25 (8148): No heartbeat from core client for 30 sec - exiting 18:05:26 (8148): No heartbeat from core client for 30 sec - exiting 18:05:27 (8148): No heartbeat from core client for 30 sec - exiting 18:05:28 (8148): No heartbeat from core client for 30 sec - exiting 18:05:29 (8148): No heartbeat from core client for 30 sec - exiting 18:05:30 (8148): No heartbeat from core client for 30 sec - exiting 18:05:31 (8148): No heartbeat from core client for 30 sec - exiting 18:05:32 (8148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:22:20 (7768): No heartbeat from core client for 30 sec - exiting 18:22:21 (7768): No heartbeat from core client for 30 sec - exiting 18:22:22 (7768): No heartbeat from core client for 30 sec - exiting 18:22:23 (7768): No heartbeat from core client for 30 sec - exiting 18:22:24 (7768): No heartbeat from core client for 30 sec - exiting 18:22:25 (7768): No heartbeat from core client for 30 sec - exiting 18:22:26 (7768): No heartbeat from core client for 30 sec - exiting 18:22:27 (7768): No heartbeat from core client for 30 sec - exiting 18:22:28 (7768): No heartbeat from core client for 30 sec - exiting 18:22:29 (7768): No heartbeat from core client for 30 sec - exiting 18:22:30 (7768): No heartbeat from core client for 30 sec - exiting 18:22:31 (7768): No heartbeat from core client for 30 sec - exiting 18:22:32 (7768): No heartbeat from core client for 30 sec - exiting 18:22:33 (7768): No heartbeat from core client for 30 sec - exiting 18:22:34 (7768): No heartbeat from core client for 30 sec - exiting 18:22:35 (7768): No heartbeat from core client for 30 sec - exiting 18:22:36 (7768): No heartbeat from core client for 30 sec - exiting 18:22:37 (7768): No heartbeat from core client for 30 sec - exiting 18:22:38 (7768): No heartbeat from core client for 30 sec - exiting 18:22:39 (7768): No heartbeat from core client for 30 sec - exiting 18:22:40 (7768): No heartbeat from core client for 30 sec - exiting 18:22:41 (7768): No heartbeat from core client for 30 sec - exiting 18:22:42 (7768): No heartbeat from core client for 30 sec - exiting 18:22:43 (7768): No heartbeat from core client for 30 sec - exiting 18:22:44 (7768): No heartbeat from core client for 30 sec - exiting 18:22:45 (7768): No heartbeat from core client for 30 sec - exiting 18:22:46 (7768): No heartbeat from core client for 30 sec - exiting 18:22:47 (7768): No heartbeat from core client for 30 sec - exiting 18:22:48 (7768): No heartbeat from core client for 30 sec - exiting 18:22:49 (7768): No heartbeat from core client for 30 sec - exiting 18:22:50 (7768): No heartbeat from core client for 30 sec - exiting 18:22:51 (7768): No heartbeat from core client for 30 sec - exiting 18:22:52 (7768): No heartbeat from core client for 30 sec - exiting 18:22:53 (7768): No heartbeat from core client for 30 sec - exiting 18:22:54 (7768): No heartbeat from core client for 30 sec - exiting 18:22:55 (7768): No heartbeat from core client for 30 sec - exiting 18:22:56 (7768): No heartbeat from core client for 30 sec - exiting 18:22:57 (7768): No heartbeat from core client for 30 sec - exiting 18:22:58 (7768): No heartbeat from core client for 30 sec - exiting 18:22:59 (7768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:52:51 (8152): No heartbeat from core client for 30 sec - exiting 17:52:52 (8152): No heartbeat from core client for 30 sec - exiting 17:52:53 (8152): No heartbeat from core client for 30 sec - exiting 17:52:54 (8152): No heartbeat from core client for 30 sec - exiting 17:52:55 (8152): No heartbeat from core client for 30 sec - exiting 17:52:56 (8152): No heartbeat from core client for 30 sec - exiting 17:52:57 (8152): No heartbeat from core client for 30 sec - exiting 17:52:58 (8152): No heartbeat from core client for 30 sec - exiting 17:52:59 (8152): No heartbeat from core client for 30 sec - exiting 17:53:00 (8152): No heartbeat from core client for 30 sec - exiting 17:53:01 (8152): No heartbeat from core client for 30 sec - exiting 17:53:02 (8152): No heartbeat from core client for 30 sec - exiting 17:53:03 (8152): No heartbeat from core client for 30 sec - exiting 17:53:04 (8152): No heartbeat from core client for 30 sec - exiting 17:53:05 (8152): No heartbeat from core client for 30 sec - exiting 17:53:06 (8152): No heartbeat from core client for 30 sec - exiting 17:53:07 (8152): No heartbeat from core client for 30 sec - exiting 17:53:08 (8152): No heartbeat from core client for 30 sec - exiting 17:53:09 (8152): No heartbeat from core client for 30 sec - exiting 17:53:10 (8152): No heartbeat from core client for 30 sec - exiting 17:53:11 (8152): No heartbeat from core client for 30 sec - exiting 17:53:12 (8152): No heartbeat from core client for 30 sec - exiting 17:53:13 (8152): No heartbeat from core client for 30 sec - exiting 17:53:14 (8152): No heartbeat from core client for 30 sec - exiting 17:53:15 (8152): No heartbeat from core client for 30 sec - exiting 17:53:16 (8152): No heartbeat from core client for 30 sec - exiting 17:53:17 (8152): No heartbeat from core client for 30 sec - exiting 17:53:18 (8152): No heartbeat from core client for 30 sec - exiting 17:53:19 (8152): No heartbeat from core client for 30 sec - exiting 17:53:20 (8152): No heartbeat from core client for 30 sec - exiting 17:53:21 (8152): No heartbeat from core client for 30 sec - exiting 17:53:22 (8152): No heartbeat from core client for 30 sec - exiting 17:53:23 (8152): No heartbeat from core client for 30 sec - exiting 17:53:24 (8152): No heartbeat from core client for 30 sec - exiting 17:53:25 (8152): No heartbeat from core client for 30 sec - exiting 17:53:26 (8152): No heartbeat from core client for 30 sec - exiting 17:53:27 (8152): No heartbeat from core client for 30 sec - exiting 17:53:28 (8152): No heartbeat from core client for 30 sec - exiting 17:53:29 (8152): No heartbeat from core client for 30 sec - exiting 17:53:30 (8152): No heartbeat from core client for 30 sec - exiting 17:53:31 (8152): No heartbeat from core client for 30 sec - exiting 17:53:32 (8152): No heartbeat from core client for 30 sec - exiting 17:53:33 (8152): No heartbeat from core client for 30 sec - exiting 17:53:34 (8152): No heartbeat from core client for 30 sec - exiting 17:53:35 (8152): No heartbeat from core client for 30 sec - exiting 17:53:36 (8152): No heartbeat from core client for 30 sec - exiting 17:53:37 (8152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7068, selfPID=7068, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:20:58 (11888): No heartbeat from core client for 30 sec - exiting 19:20:59 (11888): No heartbeat from core client for 30 sec - exiting 19:21:00 (11888): No heartbeat from core client for 30 sec - exiting 19:21:01 (11888): No heartbeat from core client for 30 sec - exiting 19:21:02 (11888): No heartbeat from core client for 30 sec - exiting 19:21:03 (11888): No heartbeat from core client for 30 sec - exiting 19:21:04 (11888): No heartbeat from core client for 30 sec - exiting 19:21:05 (11888): No heartbeat from core client for 30 sec - exiting 19:21:06 (11888): No heartbeat from core client for 30 sec - exiting 19:21:07 (11888): No heartbeat from core client for 30 sec - exiting 19:21:08 (11888): No heartbeat from core client for 30 sec - exiting 19:21:09 (11888): No heartbeat from core client for 30 sec - exiting 19:21:10 (11888): No heartbeat from core client for 30 sec - exiting 19:21:11 (11888): No heartbeat from core client for 30 sec - exiting 19:21:12 (11888): No heartbeat from core client for 30 sec - exiting 19:21:13 (11888): No heartbeat from core client for 30 sec - exiting 19:21:14 (11888): No heartbeat from core client for 30 sec - exiting 19:21:15 (11888): No heartbeat from core client for 30 sec - exiting 19:21:16 (11888): No heartbeat from core client for 30 sec - exiting 19:21:17 (11888): No heartbeat from core client for 30 sec - exiting 19:21:18 (11888): No heartbeat from core client for 30 sec - exiting 19:21:19 (11888): No heartbeat from core client for 30 sec - exiting 19:21:20 (11888): No heartbeat from core client for 30 sec - exiting 19:21:21 (11888): No heartbeat from core client for 30 sec - exiting 19:21:22 (11888): No heartbeat from core client for 30 sec - exiting 19:21:23 (11888): No heartbeat from core client for 30 sec - exiting 19:21:24 (11888): No heartbeat from core client for 30 sec - exiting 19:21:25 (11888): No heartbeat from core client for 30 sec - exiting 19:21:26 (11888): No heartbeat from core client for 30 sec - exiting 19:21:27 (11888): No heartbeat from core client for 30 sec - exiting 19:21:28 (11888): No heartbeat from core client for 30 sec - exiting 19:21:29 (11888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13324, selfPID=12604, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_w02p_2000_1_010175333_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02p_2000_1_010175333_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02p_2000_1_010175333_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02p_2000_1_010175333_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02p_2000_1_010175333_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02p_2000_1_010175333_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02p_2000_1_010175333_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Sep 2015 23:48:51	1318460	18902981	hadam3p_anz_w02p_2000_1_010175333_0	57,899	157,003	2.7117
14 Sep 2015 15:30:47	1318460	18902981	hadam3p_anz_w02p_2000_1_010175333_0	46,379	125,206	2.6996
14 Sep 2015 05:18:17	1318460	18902981	hadam3p_anz_w02p_2000_1_010175333_0	34,859	94,083	2.6990
13 Sep 2015 19:07:32	1318460	18902981	hadam3p_anz_w02p_2000_1_010175333_0	23,339	62,822	2.6917
13 Sep 2015 08:14:28	1318460	18902981	hadam3p_anz_w02p_2000_1_010175333_0	11,819	31,131	2.6340