Task 17606864

Name	hadam3p_anz_c340_2012_1_009311497_1
Workunit	9395685
Created	19 Dec 2014, 13:40:41 UTC
Sent	19 Dec 2014, 13:46:34 UTC
Report deadline	1 Dec 2015, 19:06:34 UTC
Received	27 Dec 2014, 6:28:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1302960
Run time	4 days 11 hours 5 min 4 sec
CPU time	3 days 10 hours 39 min 47 sec
Validate state	Invalid
Credit	2,497.00
Device peak FLOPS	3.30 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <stderr_txt> 01:40:06 (1300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:24 (3248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:22:48 (2400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:35:39 (1496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:05:26 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:13:37 (1728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:41:30 (4272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:28 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:49:17 (5408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:34:54 (3184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:14:58 (2724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:28:08 (5424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:41:01 (292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:51:33 (5468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:04:31 (4504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:33:15 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:46:49 (1536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:58:41 (4716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:09:03 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:14:43 (3876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:03:02 (3004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:47 (6316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:53 (7124): Can't acquire lockfile (32) - waiting 35s 11:17:32 (7124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:05 (7064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:42:53 (2708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:52:36 (6556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:18:18 (3724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:38 (2584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... G3:24:59 (5568): No heartbeat from core client for 30 sec - exiting lobal Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=1652, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 CPDN Monitor - No 'heartbeat' from BOINC... 14:12:24 (1540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:26:41 (2376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:21:18 (2568): No heartbeat from core client for 30 sec - exiting 01:21:22 (2568): No heartbeat from core client for 30 sec - exiting 01:21:23 (2568): No heartbeat from core client for 30 sec - exiting 01:21:24 (2568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:25:53 (4352): No heartbeat from core client for 30 sec - exiting 01:25:54 (4352): No heartbeat from core client for 30 sec - exiting 01:25:55 (4352): No heartbeat from core client for 30 sec - exiting 01:25:56 (4352): No heartbeat from core client for 30 sec - exiting 01:25:57 (4352): No heartbeat from core client for 30 sec - exiting 01:25:58 (4352): No heartbeat from core client for 30 sec - exiting 01:25:59 (4352): No heartbeat from core client for 30 sec - exiting 01:26:00 (4352): No heartbeat from core client for 30 sec - exiting 01:26:01 (4352): No heartbeat from core client for 30 sec - exiting 01:26:42 (4352): No heartbeat from core client for 30 sec - exiting 01:26:43 (4352): No heartbeat from core client for 30 sec - exiting 01:26:44 (4352): No heartbeat from core client for 30 sec - exiting 01:26:45 (4352): No heartbeat from core client for 30 sec - exiting 01:26:46 (4352): No heartbeat from core client for 30 sec - exiting 01:26:47 (4352): No heartbeat from core client for 30 sec - exiting 01:26:48 (4352): No heartbeat from core client for 30 sec - exiting 01:26:50 (4352): No heartbeat from core client for 30 sec - exiting 01:26:51 (4352): No heartbeat from core client for 30 sec - exiting 01:26:52 (4352): No heartbeat from core client for 30 sec - exiting 01:26:53 (4352): No heartbeat from core client for 30 sec - exiting 01:26:54 (4352): No heartbeat from core client for 30 sec - exiting 01:26:55 (4352): No heartbeat from core client for 30 sec - exiting 01:26:56 (4352): No heartbeat from core client for 30 sec - exiting 01:26:57 (4352): No heartbeat from core client for 30 sec - exiting 01:30:05 (4352): No heartbeat from core client for 30 sec - exiting 01:30:06 (4352): No heartbeat from core client for 30 sec - exiting 01:30:07 (4352): No heartbeat from core client for 30 sec - exiting 01:30:08 (4352): No heartbeat from core client for 30 sec - exiting 01:30:09 (4352): No heartbeat from core client for 30 sec - exiting 01:30:10 (4352): No heartbeat from core client for 30 sec - exiting 01:30:11 (4352): No heartbeat from core client for 30 sec - exiting 01:30:12 (4352): No heartbeat from core client for 30 sec - exiting 01:30:13 (4352): No heartbeat from core client for 30 sec - exiting 01:30:14 (4352): No heartbeat from core client for 30 sec - exiting 01:30:15 (4352): No heartbeat from core client for 30 sec - exiting 01:30:17 (4352): No heartbeat from core client for 30 sec - exiting 01:30:18 (4352): No heartbeat from core client for 30 sec - exiting 01:30:19 (4352): No heartbeat from core client for 30 sec - exiting 01:30:20 (4352): No heartbeat from core client for 30 sec - exiting 01:30:21 (4352): No heartbeat from core client for 30 sec - exiting 01:30:22 (4352): No heartbeat from core client for 30 sec - exiting 01:39:55 (4352): No heartbeat from core client for 30 sec - exiting 01:39:57 (4352): No heartbeat from core client for 30 sec - exiting 01:39:58 (4352): No heartbeat from core client for 30 sec - exiting 01:39:59 (4352): No heartbeat from core client for 30 sec - exiting 01:40:00 (4352): No heartbeat from core client for 30 sec - exiting 01:40:01 (4352): No heartbeat from core client for 30 sec - exiting 01:40:02 (4352): No heartbeat from core client for 30 sec - exiting 01:40:03 (4352): No heartbeat from core client for 30 sec - exiting 01:40:04 (4352): No heartbeat from core client for 30 sec - exiting 01:40:05 (4352): No heartbeat from core client for 30 sec - exiting 01:40:06 (4352): No heartbeat from core client for 30 sec - exiting 01:40:07 (4352): No heartbeat from core client for 30 sec - exiting 01:40:09 (4352): No heartbeat from core client for 30 sec - exiting 01:40:10 (4352): No heartbeat from core client for 30 sec - exiting 01:40:11 (4352): No heartbeat from core client for 30 sec - exiting 01:40:12 (4352): No heartbeat from core client for 30 sec - exiting 01:42:34 (4352): No heartbeat from core client for 30 sec - exiting 01:42:35 (4352): No heartbeat from core client for 30 sec - exiting 01:42:36 (4352): No heartbeat from core client for 30 sec - exiting 01:42:37 (4352): No heartbeat from core client for 30 sec - exiting 01:42:38 (4352): No heartbeat from core client for 30 sec - exiting 01:42:40 (4352): No heartbeat from core client for 30 sec - exiting 01:42:41 (4352): No heartbeat from core client for 30 sec - exiting 01:42:42 (4352): No heartbeat from core client for 30 sec - exiting 01:42:43 (4352): No heartbeat from core client for 30 sec - exiting 02:10:00 (4352): No heartbeat from core client for 30 sec - exiting 02:10:01 (4352): No heartbeat from core client for 30 sec - exiting 02:10:02 (4352): No heartbeat from core client for 30 sec - exiting 02:10:03 (4352): No heartbeat from core client for 30 sec - exiting 02:10:04 (4352): No heartbeat from core client for 30 sec - exiting 02:12:33 (4352): No heartbeat from core client for 30 sec - exiting 02:12:34 (4352): No heartbeat from core client for 30 sec - exiting 02:12:35 (4352): No heartbeat from core client for 30 sec - exiting 02:12:36 (4352): No heartbeat from core client for 30 sec - exiting 02:12:37 (4352): No heartbeat from core client for 30 sec - exiting 02:12:38 (4352): No heartbeat from core client for 30 sec - exiting 02:12:40 (4352): No heartbeat from core client for 30 sec - exiting 02:12:41 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:20:03 (2540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=2420, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2964, selfPID=2044, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_c340_2012_1_009311497_1_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c340_2012_1_009311497_1_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c340_2012_1_009311497_1_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c340_2012_1_009311497_1_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c340_2012_1_009311497_1_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c340_2012_1_009311497_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c340_2012_1_009311497_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Dec 2014 00:05:29	1302960	17606864	hadam3p_anz_c340_2012_1_009311497_1	57,899	275,536	4.7589
24 Dec 2014 11:12:09	1302960	17606864	hadam3p_anz_c340_2012_1_009311497_1	46,379	221,039	4.7659
23 Dec 2014 16:30:05	1302960	17606864	hadam3p_anz_c340_2012_1_009311497_1	34,859	166,093	4.7647
22 Dec 2014 17:58:45	1302960	17606864	hadam3p_anz_c340_2012_1_009311497_1	23,339	110,883	4.7510
21 Dec 2014 10:56:42	1302960	17606864	hadam3p_anz_c340_2012_1_009311497_1	11,819	55,625	4.7064