Task 16644073

Name	hadam3p_anz_r48c_2012_1_008735098_1
Workunit	8881076
Created	19 May 2014, 15:55:55 UTC
Sent	19 May 2014, 16:05:13 UTC
Report deadline	1 May 2015, 21:25:13 UTC
Received	20 Jun 2014, 8:39:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1262606
Run time	3 days 19 hours 15 min 8 sec
CPU time	3 days 18 hours 52 min 43 sec
Validate state	Invalid
Credit	1,503.36
Device peak FLOPS	2.79 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5712, selfPID=5232, iMonCtr=1 Model crash detected, will try to restart... 13:51:38 (4840): No heartbeat from core client for 30 sec - exiting 13:51:39 (4840): No heartbeat from core client for 30 sec - exiting 13:51:40 (4840): No heartbeat from core client for 30 sec - exiting 13:51:41 (4840): No heartbeat from core client for 30 sec - exiting 13:51:42 (4840): No heartbeat from core client for 30 sec - exiting 13:51:43 (4840): No heartbeat from core client for 30 sec - exiting 13:51:44 (4840): No heartbeat from core client for 30 sec - exiting 13:51:45 (4840): No heartbeat from core client for 30 sec - exiting 13:51:46 (4840): No heartbeat from core client for 30 sec - exiting 13:51:47 (4840): No heartbeat from core client for 30 sec - exiting 13:51:48 (4840): No heartbeat from core client for 30 sec - exiting 13:51:49 (4840): No heartbeat from core client for 30 sec - exiting 13:51:51 (4840): No heartbeat from core client for 30 sec - exiting 13:51:52 (4840): No heartbeat from core client for 30 sec - exiting 13:51:53 (4840): No heartbeat from core client for 30 sec - exiting 13:51:54 (4840): No heartbeat from core client for 30 sec - exiting 13:51:55 (4840): No heartbeat from core client for 30 sec - exiting 13:51:56 (4840): No heartbeat from core client for 30 sec - exiting 13:51:57 (4840): No heartbeat from core client for 30 sec - exiting 13:51:58 (4840): No heartbeat from core client for 30 sec - exiting 13:51:59 (4840): No heartbeat from core client for 30 sec - exiting 13:52:00 (4840): No heartbeat from core client for 30 sec - exiting 13:52:01 (4840): No heartbeat from core client for 30 sec - exiting 13:52:03 (4840): No heartbeat from core client for 30 sec - exiting 13:52:04 (4840): No heartbeat from core client for 30 sec - exiting 13:52:05 (4840): No heartbeat from core client for 30 sec - exiting 13:52:06 (4840): No heartbeat from core client for 30 sec - exiting 13:52:07 (4840): No heartbeat from core client for 30 sec - exiting 13:52:08 (4840): No heartbeat from core client for 30 sec - exiting 13:52:09 (4840): No heartbeat from core client for 30 sec - exiting 13:52:10 (4840): No heartbeat from core client for 30 sec - exiting 13:52:11 (4840): No heartbeat from core client for 30 sec - exiting 13:52:12 (4840): No heartbeat from core client for 30 sec - exiting 13:52:13 (4840): No heartbeat from core client for 30 sec - exiting 13:52:15 (4840): No heartbeat from core client for 30 sec - exiting 13:52:16 (4840): No heartbeat from core client for 30 sec - exiting 13:52:17 (4840): No heartbeat from core client for 30 sec - exiting 13:52:18 (4840): No heartbeat from core client for 30 sec - exiting 13:52:19 (4840): No heartbeat from core client for 30 sec - exiting 13:52:20 (4840): No heartbeat from core client for 30 sec - exiting 13:52:21 (4840): No heartbeat from core client for 30 sec - exiting 13:52:22 (4840): No heartbeat from core client for 30 sec - exiting 13:52:23 (4840): No heartbeat from core client for 30 sec - exiting 13:52:24 (4840): No heartbeat from core client for 30 sec - exiting 13:52:25 (4840): No heartbeat from core client for 30 sec - exiting 13:52:27 (4840): No heartbeat from core client for 30 sec - exiting 13:52:28 (4840): No heartbeat from core client for 30 sec - exiting 13:52:29 (4840): No heartbeat from core client for 30 sec - exiting 13:52:30 (4840): No heartbeat from core client for 30 sec - exiting 13:52:31 (4840): No heartbeat from core client for 30 sec - exiting 13:52:32 (4840): No heartbeat from core client for 30 sec - exiting 13:52:33 (4840): No heartbeat from core client for 30 sec - exiting 13:52:34 (4840): No heartbeat from core client for 30 sec - exiting 13:52:35 (4840): No heartbeat from core client for 30 sec - exiting 13:52:36 (4840): No heartbeat from core client for 30 sec - exiting 13:52:37 (4840): No heartbeat from core client for 30 sec - exiting 13:52:39 (4840): No heartbeat from core client for 30 sec - exiting 13:52:40 (4840): No heartbeat from core client for 30 sec - exiting 13:52:41 (4840): No heartbeat from core client for 30 sec - exiting 13:52:42 (4840): No heartbeat from core client for 30 sec - exiting 13:52:43 (4840): No heartbeat from core client for 30 sec - exiting 13:52:44 (4840): No heartbeat from core client for 30 sec - exiting 13:52:45 (4840): No heartbeat from core client for 30 sec - exiting 13:52:46 (4840): No heartbeat from core client for 30 sec - exiting 13:52:47 (4840): No heartbeat from core client for 30 sec - exiting 13:52:48 (4840): No heartbeat from core client for 30 sec - exiting 13:52:49 (4840): No heartbeat from core client for 30 sec - exiting 13:52:51 (4840): No heartbeat from core client for 30 sec - exiting 13:52:52 (4840): No heartbeat from core client for 30 sec - exiting 13:52:53 (4840): No heartbeat from core client for 30 sec - exiting 13:52:54 (4840): No heartbeat from core client for 30 sec - exiting 13:52:55 (4840): No heartbeat from core client for 30 sec - exiting 13:52:56 (4840): No heartbeat from core client for 30 sec - exiting 13:52:57 (4840): No heartbeat from core client for 30 sec - exiting 13:52:58 (4840): No heartbeat from core client for 30 sec - exiting 13:52:59 (4840): No heartbeat from core client for 30 sec - exiting 13:53:00 (4840): No heartbeat from core client for 30 sec - exiting 13:53:01 (4840): No heartbeat from core client for 30 sec - exiting 13:53:03 (4840): No heartbeat from core client for 30 sec - exiting 13:53:04 (4840): No heartbeat from core client for 30 sec - exiting 13:53:05 (4840): No heartbeat from core client for 30 sec - exiting 13:53:06 (4840): No heartbeat from core client for 30 sec - exiting 13:53:07 (4840): No heartbeat from core client for 30 sec - exiting 13:53:08 (4840): No heartbeat from core client for 30 sec - exiting 13:53:09 (4840): No heartbeat from core client for 30 sec - exiting 13:53:10 (4840): No heartbeat from core client for 30 sec - exiting 13:53:11 (4840): No heartbeat from core client for 30 sec - exiting 13:53:12 (4840): No heartbeat from core client for 30 sec - exiting 13:53:14 (4840): No heartbeat from core client for 30 sec - exiting 13:53:15 (4840): No heartbeat from core client for 30 sec - exiting 13:53:16 (4840): No heartbeat from core client for 30 sec - exiting 13:53:17 (4840): No heartbeat from core client for 30 sec - exiting 13:53:18 (4840): No heartbeat from core client for 30 sec - exiting 13:53:19 (4840): No heartbeat from core client for 30 sec - exiting 13:53:20 (4840): No heartbeat from core client for 30 sec - exiting 13:53:21 (4840): No heartbeat from core client for 30 sec - exiting 13:53:22 (4840): No heartbeat from core client for 30 sec - exiting 13:53:23 (4840): No heartbeat from core client for 30 sec - exiting 13:53:24 (4840): No heartbeat from core client for 30 sec - exiting 13:53:26 (4840): No heartbeat from core client for 30 sec - exiting 13:53:27 (4840): No heartbeat from core client for 30 sec - exiting 13:53:28 (4840): No heartbeat from core client for 30 sec - exiting 13:53:29 (4840): No heartbeat from core client for 30 sec - exiting 13:53:30 (4840): No heartbeat from core client for 30 sec - exiting 13:53:31 (4840): No heartbeat from core client for 30 sec - exiting 13:53:32 (4840): No heartbeat from core client for 30 sec - exiting 13:53:33 (4840): No heartbeat from core client for 30 sec - exiting 13:53:34 (4840): No heartbeat from core client for 30 sec - exiting 13:53:35 (4840): No heartbeat from core client for 30 sec - exiting 13:53:36 (4840): No heartbeat from core client for 30 sec - exiting 13:53:38 (4840): No heartbeat from core client for 30 sec - exiting 13:53:39 (4840): No heartbeat from core client for 30 sec - exiting 13:53:40 (4840): No heartbeat from core client for 30 sec - exiting 13:53:41 (4840): No heartbeat from core client for 30 sec - exiting 13:53:42 (4840): No heartbeat from core client for 30 sec - exiting 13:53:43 (4840): No heartbeat from core client for 30 sec - exiting 13:53:44 (4840): No heartbeat from core client for 30 sec - exiting 13:53:45 (4840): No heartbeat from core client for 30 sec - exiting 13:53:46 (4840): No heartbeat from core client for 30 sec - exiting 13:53:47 (4840): No heartbeat from core client for 30 sec - exiting 13:53:48 (4840): No heartbeat from core client for 30 sec - exiting 13:53:50 (4840): No heartbeat from core client for 30 sec - exiting 13:53:51 (4840): No heartbeat from core client for 30 sec - exiting 13:53:52 (4840): No heartbeat from core client for 30 sec - exiting 13:53:53 (4840): No heartbeat from core client for 30 sec - exiting 13:53:54 (4840): No heartbeat from core client for 30 sec - exiting 13:53:55 (4840): No heartbeat from core client for 30 sec - exiting 13:53:56 (4840): No heartbeat from core client for 30 sec - exiting 13:53:57 (4840): No heartbeat from core client for 30 sec - exiting 13:53:58 (4840): No heartbeat from core client for 30 sec - exiting 13:53:59 (4840): No heartbeat from core client for 30 sec - exiting 13:54:00 (4840): No heartbeat from core client for 30 sec - exiting 13:54:02 (4840): No heartbeat from core client for 30 sec - exiting 13:54:03 (4840): No heartbeat from core client for 30 sec - exiting 13:54:04 (4840): No heartbeat from core client for 30 sec - exiting 13:54:05 (4840): No heartbeat from core client for 30 sec - exiting 13:54:06 (4840): No heartbeat from core client for 30 sec - exiting 13:54:07 (4840): No heartbeat from core client for 30 sec - exiting 13:54:08 (4840): No heartbeat from core client for 30 sec - exiting 13:54:09 (4840): No heartbeat from core client for 30 sec - exiting 13:54:10 (4840): No heartbeat from core client for 30 sec - exiting 13:54:11 (4840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:54:12 (4840): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5348, selfPID=1624, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3904, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5936, iMonCtr=2 Model crash detected, will try to restart... 08:24:39 (3284): No heartbeat from core client for 30 sec - exiting 08:24:40 (3284): No heartbeat from core client for 30 sec - exiting 08:24:42 (3284): No heartbeat from core client for 30 sec - exiting 08:24:43 (3284): No heartbeat from core client for 30 sec - exiting 08:24:44 (3284): No heartbeat from core client for 30 sec - exiting 08:24:45 (3284): No heartbeat from core client for 30 sec - exiting 08:24:46 (3284): No heartbeat from core client for 30 sec - exiting 08:24:47 (3284): No heartbeat from core client for 30 sec - exiting 08:24:48 (3284): No heartbeat from core client for 30 sec - exiting 08:24:49 (3284): No heartbeat from core client for 30 sec - exiting 08:24:50 (3284): No heartbeat from core client for 30 sec - exiting 08:24:51 (3284): No heartbeat from core client for 30 sec - exiting 08:24:52 (3284): No heartbeat from core client for 30 sec - exiting 08:24:53 (3284): No heartbeat from core client for 30 sec - exiting 08:24:54 (3284): No heartbeat from core client for 30 sec - exiting 08:24:55 (3284): No heartbeat from core client for 30 sec - exiting 08:24:56 (3284): No heartbeat from core client for 30 sec - exiting 08:24:58 (3284): No heartbeat from core client for 30 sec - exiting 08:24:59 (3284): No heartbeat from core client for 30 sec - exiting 08:25:00 (3284): No heartbeat from core client for 30 sec - exiting 08:25:01 (3284): No heartbeat from core client for 30 sec - exiting 08:25:02 (3284): No heartbeat from core client for 30 sec - exiting 08:25:03 (3284): No heartbeat from core client for 30 sec - exiting 08:25:04 (3284): No heartbeat from core client for 30 sec - exiting 08:25:05 (3284): No heartbeat from core client for 30 sec - exiting 08:25:06 (3284): No heartbeat from core client for 30 sec - exiting 08:25:07 (3284): No heartbeat from core client for 30 sec - exiting 08:25:08 (3284): No heartbeat from core client for 30 sec - exiting 08:25:10 (3284): No heartbeat from core client for 30 sec - exiting 08:25:11 (3284): No heartbeat from core client for 30 sec - exiting 08:25:12 (3284): No heartbeat from core client for 30 sec - exiting 08:25:13 (3284): No heartbeat from core client for 30 sec - exiting 08:25:14 (3284): No heartbeat from core client for 30 sec - exiting 08:25:15 (3284): No heartbeat from core client for 30 sec - exiting 08:25:16 (3284): No heartbeat from core client for 30 sec - exiting 08:25:17 (3284): No heartbeat from core client for 30 sec - exiting 08:25:18 (3284): No heartbeat from core client for 30 sec - exiting 08:25:19 (3284): No heartbeat from core client for 30 sec - exiting 08:25:20 (3284): No heartbeat from core client for 30 sec - exiting 08:25:21 (3284): No heartbeat from core client for 30 sec - exiting 08:25:23 (3284): No heartbeat from core client for 30 sec - exiting 08:25:24 (3284): No heartbeat from core client for 30 sec - exiting 08:25:25 (3284): No heartbeat from core client for 30 sec - exiting 08:25:26 (3284): No heartbeat from core client for 30 sec - exiting 08:25:27 (3284): No heartbeat from core client for 30 sec - exiting 08:25:28 (3284): No heartbeat from core client for 30 sec - exiting 08:25:29 (3284): No heartbeat from core client for 30 sec - exiting 08:25:30 (3284): No heartbeat from core client for 30 sec - exiting 08:25:31 (3284): No heartbeat from core client for 30 sec - exiting 08:25:32 (3284): No heartbeat from core client for 30 sec - exiting 08:25:33 (3284): No heartbeat from core client for 30 sec - exiting 08:25:35 (3284): No heartbeat from core client for 30 sec - exiting 08:25:36 (3284): No heartbeat from core client for 30 sec - exiting 08:25:37 (3284): No heartbeat from core client for 30 sec - exiting 08:25:38 (3284): No heartbeat from core client for 30 sec - exiting 08:25:39 (3284): No heartbeat from core client for 30 sec - exiting 08:25:40 (3284): No heartbeat from core client for 30 sec - exiting 08:25:41 (3284): No heartbeat from core client for 30 sec - exiting 08:25:42 (3284): No heartbeat from core client for 30 sec - exiting 08:25:43 (3284): No heartbeat from core client for 30 sec - exiting 08:25:44 (3284): No heartbeat from core client for 30 sec - exiting 08:25:45 (3284): No heartbeat from core client for 30 sec - exiting 08:25:47 (3284): No heartbeat from core client for 30 sec - exiting 08:25:48 (3284): No heartbeat from core client for 30 sec - exiting 08:25:49 (3284): No heartbeat from core client for 30 sec - exiting 08:25:50 (3284): No heartbeat from core client for 30 sec - exiting 08:25:51 (3284): No heartbeat from core client for 30 sec - exiting 08:25:52 (3284): No heartbeat from core client for 30 sec - exiting 08:26:26 (3284): No heartbeat from core client for 30 sec - exiting 08:26:27 (3284): No heartbeat from core client for 30 sec - exiting 08:26:28 (3284): No heartbeat from core client for 30 sec - exiting 08:26:29 (3284): No heartbeat from core client for 30 sec - exiting 08:26:30 (3284): No heartbeat from core client for 30 sec - exiting 08:26:31 (3284): No heartbeat from core client for 30 sec - exiting 08:26:32 (3284): No heartbeat from core client for 30 sec - exiting 08:26:33 (3284): No heartbeat from core client for 30 sec - exiting 08:26:34 (3284): No heartbeat from core client for 30 sec - exiting 08:26:36 (3284): No heartbeat from core client for 30 sec - exiting 08:26:37 (3284): No heartbeat from core client for 30 sec - exiting 08:26:38 (3284): No heartbeat from core client for 30 sec - exiting 08:26:39 (3284): No heartbeat from core client for 30 sec - exiting 08:26:40 (3284): No heartbeat from core client for 30 sec - exiting 08:26:41 (3284): No heartbeat from core client for 30 sec - exiting 08:26:42 (3284): No heartbeat from core client for 30 sec - exiting 08:26:43 (3284): No heartbeat from core client for 30 sec - exiting 08:26:44 (3284): No heartbeat from core client for 30 sec - exiting 08:26:45 (3284): No heartbeat from core client for 30 sec - exiting 08:26:46 (3284): No heartbeat from core client for 30 sec - exiting 08:26:48 (3284): No heartbeat from core client for 30 sec - exiting 08:26:49 (3284): No heartbeat from core client for 30 sec - exiting 08:26:50 (3284): No heartbeat from core client for 30 sec - exiting 08:26:51 (3284): No heartbeat from core client for 30 sec - exiting 08:26:52 (3284): No heartbeat from core client for 30 sec - exiting 08:26:53 (3284): No heartbeat from core client for 30 sec - exiting 08:26:54 (3284): No heartbeat from core client for 30 sec - exiting 08:26:55 (3284): No heartbeat from core client for 30 sec - exiting 08:26:56 (3284): No heartbeat from core client for 30 sec - exiting 08:26:57 (3284): No heartbeat from core client for 30 sec - exiting 08:26:58 (3284): No heartbeat from core client for 30 sec - exiting 08:27:00 (3284): No heartbeat from core client for 30 sec - exiting 08:27:01 (3284): No heartbeat from core client for 30 sec - exiting 08:27:02 (3284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:28:10 (5592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=3440, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5572, selfPID=4028, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_r48c_2012_1_008735098_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Jun 2014 12:52:13	1262606	16644073	hadam3p_anz_r48c_2012_1_008735098_1	34,859	326,232	9.3586
12 Jun 2014 07:59:18	1262606	16644073	hadam3p_anz_r48c_2012_1_008735098_1	23,339	216,922	9.2944
27 May 2014 16:21:57	1262606	16644073	hadam3p_anz_r48c_2012_1_008735098_1	11,819	110,725	9.3684