Task 18294558

Name	hadam3p_anz_m3cw_2013_1_009740632_0
Workunit	9811630
Created	9 Apr 2015, 11:28:39 UTC
Sent	9 Apr 2015, 21:37:14 UTC
Report deadline	22 Mar 2016, 2:57:14 UTC
Received	14 Apr 2015, 21:21:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1212655
Run time	2 days 4 hours 44 min 7 sec
CPU time	1 days 21 hours 20 min 20 sec
Validate state	Invalid
Credit	1,006.54
Device peak FLOPS	2.41 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <stderr_txt> 05:48:09 (4300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:55:50 (6684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7356, selfPID=7356, iMonCtr=2 06:08:12 (7604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:13:55 (10040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:48:37 (8664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:48:38 (8664): No heartbeat from core client for 30 sec - exiting 07:54:12 (7524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:54:13 (7524): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 07:42:34 (9464): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 07:57:42 (8100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:57:44 (8100): No heartbeat from core client for 30 sec - exiting 07:57:45 (8100): No heartbeat from core client for 30 sec - exiting 07:57:46 (8100): No heartbeat from core client for 30 sec - exiting 07:57:47 (8100): No heartbeat from core client for 30 sec - exiting 07:57:48 (8100): No heartbeat from core client for 30 sec - exiting 07:57:49 (8100): No heartbeat from core client for 30 sec - exiting 07:57:50 (8100): No heartbeat from core client for 30 sec - exiting 07:57:51 (8100): No heartbeat from core client for 30 sec - exiting 07:57:52 (8100): No heartbeat from core client for 30 sec - exiting 07:57:53 (8100): No heartbeat from core client for 30 sec - exiting 07:57:54 (8100): No heartbeat from core client for 30 sec - exiting 08:07:44 (10808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:07:45 (10808): No heartbeat from core client for 30 sec - exiting 08:07:47 (10808): No heartbeat from core client for 30 sec - exiting 08:12:44 (6896): No heartbeat from core client for 30 sec - exiting 08:12:45 (6896): No heartbeat from core client for 30 sec - exiting 08:12:46 (6896): No heartbeat from core client for 30 sec - exiting 08:12:47 (6896): No heartbeat from core client for 30 sec - exiting 08:13:19 (6896): No heartbeat from core client for 30 sec - exiting 08:13:20 (6896): No heartbeat from core client for 30 sec - exiting 08:13:21 (6896): No heartbeat from core client for 30 sec - exiting 08:13:22 (6896): No heartbeat from core client for 30 sec - exiting 08:13:23 (6896): No heartbeat from core client for 30 sec - exiting 08:13:24 (6896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:13:25 (6896): No heartbeat from core client for 30 sec - exiting 08:16:21 (7868): No heartbeat from core client for 30 sec - exiting 08:16:22 (7868): No heartbeat from core client for 30 sec - exiting 08:16:23 (7868): No heartbeat from core client for 30 sec - exiting 08:16:24 (7868): No heartbeat from core client for 30 sec - exiting 08:16:25 (7868): No heartbeat from core client for 30 sec - exiting 08:16:26 (7868): No heartbeat from core client for 30 sec - exiting 08:16:27 (7868): No heartbeat from core client for 30 sec - exiting 08:16:28 (7868): No heartbeat from core client for 30 sec - exiting 08:16:29 (7868): No heartbeat from core client for 30 sec - exiting 08:16:30 (7868): No heartbeat from core client for 30 sec - exiting 08:16:31 (7868): No heartbeat from core client for 30 sec - exiting 08:16:32 (7868): No heartbeat from core client for 30 sec - exiting 08:16:33 (7868): No heartbeat from core client for 30 sec - exiting 08:16:34 (7868): No heartbeat from core client for 30 sec - exiting 08:16:35 (7868): No heartbeat from core client for 30 sec - exiting 08:16:36 (7868): No heartbeat from core client for 30 sec - exiting 08:16:37 (7868): No heartbeat from core client for 30 sec - exiting 08:16:38 (7868): No heartbeat from core client for 30 sec - exiting 08:16:39 (7868): No heartbeat from core client for 30 sec - exiting 08:16:40 (7868): No heartbeat from core client for 30 sec - exiting 08:16:41 (7868): No heartbeat from core client for 30 sec - exiting 08:16:42 (7868): No heartbeat from core client for 30 sec - exiting 08:16:43 (7868): No heartbeat from core client for 30 sec - exiting 08:16:44 (7868): No heartbeat from core client for 30 sec - exiting 08:16:45 (7868): No heartbeat from core client for 30 sec - exiting 08:16:46 (7868): No heartbeat from core client for 30 sec - exiting 08:16:47 (7868): No heartbeat from core client for 30 sec - exiting 08:16:48 (7868): No heartbeat from core client for 30 sec - exiting 08:16:49 (7868): No heartbeat from core client for 30 sec - exiting 08:16:50 (7868): No heartbeat from core client for 30 sec - exiting 08:16:51 (7868): No heartbeat from core client for 30 sec - exiting 08:16:52 (7868): No heartbeat from core client for 30 sec - exiting 08:16:53 (7868): No heartbeat from core client for 30 sec - exiting 08:16:54 (7868): No heartbeat from core client for 30 sec - exiting 08:16:55 (7868): No heartbeat from core client for 30 sec - exiting 08:16:56 (7868): No heartbeat from core client for 30 sec - exiting 08:16:57 (7868): No heartbeat from core client for 30 sec - exiting 08:16:58 (7868): No heartbeat from core client for 30 sec - exiting 08:16:59 (7868): No heartbeat from core client for 30 sec - exiting 08:17:00 (7868): No heartbeat from core client for 30 sec - exiting 08:17:01 (7868): No heartbeat from core client for 30 sec - exiting 08:17:02 (7868): No heartbeat from core client for 30 sec - exiting 08:17:03 (7868): No heartbeat from core client for 30 sec - exiting 08:17:04 (7868): No heartbeat from core client for 30 sec - exiting 08:17:05 (7868): No heartbeat from core client for 30 sec - exiting 08:17:06 (7868): No heartbeat from core client for 30 sec - exiting 08:17:07 (7868): No heartbeat from core client for 30 sec - exiting 08:17:08 (7868): No heartbeat from core client for 30 sec - exiting 08:17:09 (7868): No heartbeat from core client for 30 sec - exiting 08:17:10 (7868): No heartbeat from core client for 30 sec - exiting 08:17:11 (7868): No heartbeat from core client for 30 sec - exiting 08:17:12 (7868): No heartbeat from core client for 30 sec - exiting 08:17:13 (7868): No heartbeat from core client for 30 sec - exiting 08:17:14 (7868): No heartbeat from core client for 30 sec - exiting 08:17:15 (7868): No heartbeat from core client for 30 sec - exiting 08:17:16 (7868): No heartbeat from core client for 30 sec - exiting 08:17:17 (7868): No heartbeat from core client for 30 sec - exiting 08:17:18 (7868): No heartbeat from core client for 30 sec - exiting 08:17:19 (7868): No heartbeat from core client for 30 sec - exiting 08:17:20 (7868): No heartbeat from core client for 30 sec - exiting 08:17:21 (7868): No heartbeat from core client for 30 sec - exiting 08:17:22 (7868): No heartbeat from core client for 30 sec - exiting 08:17:23 (7868): No heartbeat from core client for 30 sec - exiting 08:17:24 (7868): No heartbeat from core client for 30 sec - exiting 08:17:25 (7868): No heartbeat from core client for 30 sec - exiting 08:17:26 (7868): No heartbeat from core client for 30 sec - exiting 08:17:27 (7868): No heartbeat from core client for 30 sec - exiting 08:17:28 (7868): No heartbeat from core client for 30 sec - exiting 08:17:29 (7868): No heartbeat from core client for 30 sec - exiting 08:17:30 (7868): No heartbeat from core client for 30 sec - exiting 08:17:31 (7868): No heartbeat from core client for 30 sec - exiting 08:17:32 (7868): No heartbeat from core client for 30 sec - exiting 08:17:33 (7868): No heartbeat from core client for 30 sec - exiting 08:17:34 (7868): No heartbeat from core client for 30 sec - exiting 08:17:35 (7868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:22:28 (3896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:16 (2872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:17 (2872): No heartbeat from core client for 30 sec - exiting 08:56:18 (2872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:51:51 (11716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:14:34 (10776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:14:35 (10776): No heartbeat from core client for 30 sec - exiting 12:14:36 (10776): No heartbeat from core client for 30 sec - exiting 12:14:37 (10776): No heartbeat from core client for 30 sec - exiting 12:14:38 (10776): No heartbeat from core client for 30 sec - exiting 12:14:39 (10776): No heartbeat from core client for 30 sec - exiting 12:14:40 (10776): No heartbeat from core client for 30 sec - exiting 14:02:03 (10076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:22:16 (6528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:25:07 (12088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:25:08 (12088): No heartbeat from core client for 30 sec - exiting 15:25:09 (12088): No heartbeat from core client for 30 sec - exiting 15:25:10 (12088): No heartbeat from core client for 30 sec - exiting 15:25:11 (12088): No heartbeat from core client for 30 sec - exiting 15:25:12 (12088): No heartbeat from core client for 30 sec - exiting 15:25:13 (12088): No heartbeat from core client for 30 sec - exiting 16:51:17 (8448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:18 (8448): No heartbeat from core client for 30 sec - exiting 16:51:19 (8448): No heartbeat from core client for 30 sec - exiting 16:51:20 (8448): No heartbeat from core client for 30 sec - exiting 16:51:21 (8448): No heartbeat from core client for 30 sec - exiting 16:52:08 (6784): No heartbeat from core client for 30 sec - exiting 16:52:09 (6784): No heartbeat from core client for 30 sec - exiting 16:52:10 (6784): No heartbeat from core client for 30 sec - exiting 16:52:11 (6784): No heartbeat from core client for 30 sec - exiting 16:52:12 (6784): No heartbeat from core client for 30 sec - exiting 16:52:13 (6784): No heartbeat from core client for 30 sec - exiting 16:52:14 (6784): No heartbeat from core client for 30 sec - exiting 16:52:15 (6784): No heartbeat from core client for 30 sec - exiting 16:52:16 (6784): No heartbeat from core client for 30 sec - exiting 16:52:17 (6784): No heartbeat from core client for 30 sec - exiting 16:52:18 (6784): No heartbeat from core client for 30 sec - exiting 16:52:19 (6784): No heartbeat from core client for 30 sec - exiting 16:52:20 (6784): No heartbeat from core client for 30 sec - exiting 16:52:21 (6784): No heartbeat from core client for 30 sec - exiting 16:52:22 (6784): No heartbeat from core client for 30 sec - exiting 16:52:23 (6784): No heartbeat from core client for 30 sec - exiting 16:52:24 (6784): No heartbeat from core client for 30 sec - exiting 16:52:25 (6784): No heartbeat from core client for 30 sec - exiting 16:52:26 (6784): No heartbeat from core client for 30 sec - exiting 16:52:27 (6784): No heartbeat from core client for 30 sec - exiting 16:52:28 (6784): No heartbeat from core client for 30 sec - exiting 16:52:29 (6784): No heartbeat from core client for 30 sec - exiting 16:52:30 (6784): No heartbeat from core client for 30 sec - exiting 16:52:31 (6784): No heartbeat from core client for 30 sec - exiting 16:52:32 (6784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:43:21 (2684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:45 (4492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:39:21 (9680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:06:23 (9468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:09:15 (3368): No heartbeat from core client for 30 sec - exiting 21:09:16 (3368): No heartbeat from core client for 30 sec - exiting 21:09:17 (3368): No heartbeat from core client for 30 sec - exiting 21:09:18 (3368): No heartbeat from core client for 30 sec - exiting 21:09:19 (3368): No heartbeat from core client for 30 sec - exiting 21:09:20 (3368): No heartbeat from core client for 30 sec - exiting 21:09:21 (3368): No heartbeat from core client for 30 sec - exiting 21:09:22 (3368): No heartbeat from core client for 30 sec - exiting 21:09:23 (3368): No heartbeat from core client for 30 sec - exiting 21:09:24 (3368): No heartbeat from core client for 30 sec - exiting 21:09:25 (3368): No heartbeat from core client for 30 sec - exiting 21:09:26 (3368): No heartbeat from core client for 30 sec - exiting 21:09:27 (3368): No heartbeat from core client for 30 sec - exiting 21:09:28 (3368): No heartbeat from core client for 30 sec - exiting 21:09:29 (3368): No heartbeat from core client for 30 sec - exiting 21:09:30 (3368): No heartbeat from core client for 30 sec - exiting 21:09:31 (3368): No heartbeat from core client for 30 sec - exiting 21:09:32 (3368): No heartbeat from core client for 30 sec - exiting 21:09:33 (3368): No heartbeat from core client for 30 sec - exiting 21:09:34 (3368): No heartbeat from core client for 30 sec - exiting 21:09:35 (3368): No heartbeat from core client for 30 sec - exiting 21:09:36 (3368): No heartbeat from core client for 30 sec - exiting 21:09:37 (3368): No heartbeat from core client for 30 sec - exiting 21:09:38 (3368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:44:44 (6536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:23:03 (6588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:33:54 (7128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8364, selfPID=8364, iMonCtr=2 03:07:00 (7616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:23:41 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:29:13 (8336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:32:56 (5696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:54 (8632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:12:54 (10380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:16:31 (4056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:06 (7988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:25:02 (6560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional 07:36:16 (4136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:56:03 (12528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:56:04 (12528): No heartbeat from core client for 30 sec - exiting 09:54:35 (12364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:36 (12364): No heartbeat from core client for 30 sec - exiting 09:54:37 (12364): No heartbeat from core client for 30 sec - exiting 09:54:38 (12364): No heartbeat from core client for 30 sec - exiting 11:00:44 (12412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7900, selfPID=7900, iMonCtr=2 11:27:18 (11556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:27:19 (11556): No heartbeat from core client for 30 sec - exiting 11:27:20 (11556): No heartbeat from core client for 30 sec - exiting 11:27:21 (11556): No heartbeat from core client for 30 sec - exiting 11:27:22 (11556): No heartbeat from core client for 30 sec - exiting 12:04:51 (6728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:04:52 (6728): No heartbeat from core client for 30 sec - exiting 12:04:53 (6728): No heartbeat from core client for 30 sec - exiting 12:04:54 (6728): No heartbeat from core client for 30 sec - exiting 12:13:36 (12552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:13:37 (12552): No heartbeat from core client for 30 sec - exiting 12:13:38 (12552): No heartbeat from core client for 30 sec - exiting 13:18:13 (13236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:27:04 (10456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:35:40 (11364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:13:52 (7856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:13:53 (7856): No heartbeat from core client for 30 sec - exiting 14:13:54 (7856): No heartbeat from core client for 30 sec - exiting 14:13:55 (7856): No heartbeat from core client for 30 sec - exiting 14:13:56 (7856): No heartbeat from core client for 30 sec - exiting 14:13:57 (7856): No heartbeat from core client for 30 sec - exiting 14:18:05 (10580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:34:07 (7404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:21 (8892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:22 (8892): No heartbeat from core client for 30 sec - exiting 14:44:23 (8892): No heartbeat from core client for 30 sec - exiting 15:14:03 (2888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:14:04 (2888): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=11728, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10056, selfPID=10056, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10056, selfPID=7868, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m3cw_2013_1_009740632_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Apr 2015 21:25:45	1212655	18294558	hadam3p_anz_m3cw_2013_1_009740632_0	23,339	158,078	6.7731
12 Apr 2015 06:44:26	1212655	18294558	hadam3p_anz_m3cw_2013_1_009740632_0	11,819	79,564	6.7319