Task 16418329

Name	hadam3p_anz_nac7_2012_1_008601015_0
Workunit	8747527
Created	26 Mar 2014, 19:20:18 UTC
Sent	27 Mar 2014, 7:15:41 UTC
Report deadline	9 Mar 2015, 12:35:41 UTC
Received	29 Aug 2014, 9:32:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1205870
Run time	4 days 1 hours 13 min 47 sec
CPU time	3 days 22 hours 53 min 49 sec
Validate state	Invalid
Credit	2,000.18
Device peak FLOPS	3.15 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:40:28 (2972): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6108, selfPID=6108, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 06:59:33 (2148): No heartbeat from core client for 30 sec - exiting 06:59:34 (2148): No heartbeat from core client for 30 sec - exiting 06:59:35 (2148): No heartbeat from core client for 30 sec - exiting 06:59:36 (2148): No heartbeat from core client for 30 sec - exiting 06:59:37 (2148): No heartbeat from core client for 30 sec - exiting 06:59:39 (2148): No heartbeat from core client for 30 sec - exiting 06:59:40 (2148): No heartbeat from core client for 30 sec - exiting 06:59:41 (2148): No heartbeat from core client for 30 sec - exiting 06:59:42 (2148): No heartbeat from core client for 30 sec - exiting 06:59:43 (2148): No heartbeat from core client for 30 sec - exiting 06:59:44 (2148): No heartbeat from core client for 30 sec - exiting 06:59:45 (2148): No heartbeat from core client for 30 sec - exiting 06:59:46 (2148): No heartbeat from core client for 30 sec - exiting 06:59:47 (2148): No heartbeat from core client for 30 sec - exiting 06:59:48 (2148): No heartbeat from core client for 30 sec - exiting 06:59:49 (2148): No heartbeat from core client for 30 sec - exiting 06:59:51 (2148): No heartbeat from core client for 30 sec - exiting 06:59:52 (2148): No heartbeat from core client for 30 sec - exiting 06:59:53 (2148): No heartbeat from core client for 30 sec - exiting 06:59:54 (2148): No heartbeat from core client for 30 sec - exiting 06:59:55 (2148): No heartbeat from core client for 30 sec - exiting 06:59:56 (2148): No heartbeat from core client for 30 sec - exiting 06:59:57 (2148): No heartbeat from core client for 30 sec - exiting 06:59:58 (2148): No heartbeat from core client for 30 sec - exiting 06:59:59 (2148): No heartbeat from core client for 30 sec - exiting 07:00:00 (2148): No heartbeat from core client for 30 sec - exiting 07:00:02 (2148): No heartbeat from core client for 30 sec - exiting 07:00:03 (2148): No heartbeat from core client for 30 sec - exiting 07:00:04 (2148): No heartbeat from core client for 30 sec - exiting 07:00:05 (2148): No heartbeat from core client for 30 sec - exiting 07:00:06 (2148): No heartbeat from core client for 30 sec - exiting 07:00:07 (2148): No heartbeat from core client for 30 sec - exiting 07:00:08 (2148): No heartbeat from core client for 30 sec - exiting 07:00:09 (2148): No heartbeat from core client for 30 sec - exiting 07:00:10 (2148): No heartbeat from core client for 30 sec - exiting 07:00:11 (2148): No heartbeat from core client for 30 sec - exiting 07:00:12 (2148): No heartbeat from core client for 30 sec - exiting 07:00:14 (2148): No heartbeat from core client for 30 sec - exiting 07:00:15 (2148): No heartbeat from core client for 30 sec - exiting 07:00:16 (2148): No heartbeat from core client for 30 sec - exiting 07:00:17 (2148): No heartbeat from core client for 30 sec - exiting 07:00:18 (2148): No heartbeat from core client for 30 sec - exiting 07:00:19 (2148): No heartbeat from core client for 30 sec - exiting 07:00:20 (2148): No heartbeat from core client for 30 sec - exiting 07:00:21 (2148): No heartbeat from core client for 30 sec - exiting 07:00:22 (2148): No heartbeat from core client for 30 sec - exiting 07:00:23 (2148): No heartbeat from core client for 30 sec - exiting 07:00:25 (2148): No heartbeat from core client for 30 sec - exiting 07:00:26 (2148): No heartbeat from core client for 30 sec - exiting 07:00:27 (2148): No heartbeat from core client for 30 sec - exiting 07:00:28 (2148): No heartbeat from core client for 30 sec - exiting 07:00:29 (2148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 06:51:14 (1068): No heartbeat from core client for 30 sec - exiting 06:51:15 (1068): No heartbeat from core client for 30 sec - exiting 06:51:16 (1068): No heartbeat from core client for 30 sec - exiting 06:51:17 (1068): No heartbeat from core client for 30 sec - exiting 06:51:19 (1068): No heartbeat from core client for 30 sec - exiting 06:51:20 (1068): No heartbeat from core client for 30 sec - exiting 06:51:21 (1068): No heartbeat from core client for 30 sec - exiting 06:51:22 (1068): No heartbeat from core client for 30 sec - exiting 06:51:23 (1068): No heartbeat from core client for 30 sec - exiting 06:51:24 (1068): No heartbeat from core client for 30 sec - exiting 06:51:25 (1068): No heartbeat from core client for 30 sec - exiting 06:51:26 (1068): No heartbeat from core client for 30 sec - exiting 06:51:27 (1068): No heartbeat from core client for 30 sec - exiting 06:51:28 (1068): No heartbeat from core client for 30 sec - exiting 06:51:29 (1068): No heartbeat from core client for 30 sec - exiting 06:51:31 (1068): No heartbeat from core client for 30 sec - exiting 06:51:32 (1068): No heartbeat from core client for 30 sec - exiting 06:51:33 (1068): No heartbeat from core client for 30 sec - exiting 06:51:34 (1068): No heartbeat from core client for 30 sec - exiting 06:51:35 (1068): No heartbeat from core client for 30 sec - exiting 06:51:36 (1068): No heartbeat from core client for 30 sec - exiting 06:51:37 (1068): No heartbeat from core client for 30 sec - exiting 06:51:38 (1068): No heartbeat from core client for 30 sec - exiting 06:51:39 (1068): No heartbeat from core client for 30 sec - exiting 06:51:40 (1068): No heartbeat from core client for 30 sec - exiting 06:51:42 (1068): No heartbeat from core client for 30 sec - exiting 06:51:43 (1068): No heartbeat from core client for 30 sec - exiting 06:51:44 (1068): No heartbeat from core client for 30 sec - exiting 06:51:45 (1068): No heartbeat from core client for 30 sec - exiting 06:51:46 (1068): No heartbeat from core client for 30 sec - exiting 06:51:47 (1068): No heartbeat from core client for 30 sec - exiting 06:51:48 (1068): No heartbeat from core client for 30 sec - exiting 06:51:49 (1068): No heartbeat from core client for 30 sec - exiting 06:51:50 (1068): No heartbeat from core client for 30 sec - exiting 06:51:51 (1068): No heartbeat from core client for 30 sec - exiting 06:51:52 (1068): No heartbeat from core client for 30 sec - exiting 06:51:54 (1068): No heartbeat from core client for 30 sec - exiting 06:51:55 (1068): No heartbeat from core client for 30 sec - exiting 06:51:56 (1068): No heartbeat from core client for 30 sec - exiting 06:51:57 (1068): No heartbeat from core client for 30 sec - exiting 06:51:58 (1068): No heartbeat from core client for 30 sec - exiting 06:51:59 (1068): No heartbeat from core client for 30 sec - exiting 06:52:00 (1068): No heartbeat from core client for 30 sec - exiting 06:52:01 (1068): No heartbeat from core client for 30 sec - exiting 06:52:02 (1068): No heartbeat from core client for 30 sec - exiting 06:52:03 (1068): No heartbeat from core client for 30 sec - exiting 06:52:04 (1068): No heartbeat from core client for 30 sec - exiting 06:52:06 (1068): No heartbeat from core client for 30 sec - exiting 06:52:07 (1068): No heartbeat from core client for 30 sec - exiting 06:52:08 (1068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:52:09 (1068): No heartbeat from core client for 30 sec - exiting 06:52:10 (1068): No heartbeat from core client for 30 sec - exiting 06:52:11 (1068): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 06:55:08 (572): No heartbeat from core client for 30 sec - exiting 06:55:09 (572): No heartbeat from core client for 30 sec - exiting 06:55:10 (572): No heartbeat from core client for 30 sec - exiting 06:55:11 (572): No heartbeat from core client for 30 sec - exiting 06:55:13 (572): No heartbeat from core client for 30 sec - exiting 06:55:14 (572): No heartbeat from core client for 30 sec - exiting 06:55:15 (572): No heartbeat from core client for 30 sec - exiting 06:55:16 (572): No heartbeat from core client for 30 sec - exiting 06:55:17 (572): No heartbeat from core client for 30 sec - exiting 06:55:18 (572): No heartbeat from core client for 30 sec - exiting 06:55:19 (572): No heartbeat from core client for 30 sec - exiting 06:55:20 (572): No heartbeat from core client for 30 sec - exiting 06:55:21 (572): No heartbeat from core client for 30 sec - exiting 06:55:22 (572): No heartbeat from core client for 30 sec - exiting 06:55:23 (572): No heartbeat from core client for 30 sec - exiting 06:55:25 (572): No heartbeat from core client for 30 sec - exiting 06:55:26 (572): No heartbeat from core client for 30 sec - exiting 06:55:27 (572): No heartbeat from core client for 30 sec - exiting 06:55:28 (572): No heartbeat from core client for 30 sec - exiting 06:55:29 (572): No heartbeat from core client for 30 sec - exiting 06:55:30 (572): No heartbeat from core client for 30 sec - exiting 06:55:31 (572): No heartbeat from core client for 30 sec - exiting 06:55:32 (572): No heartbeat from core client for 30 sec - exiting 06:55:33 (572): No heartbeat from core client for 30 sec - exiting 06:55:34 (572): No heartbeat from core client for 30 sec - exiting 06:55:36 (572): No heartbeat from core client for 30 sec - exiting 06:55:37 (572): No heartbeat from core client for 30 sec - exiting 06:55:38 (572): No heartbeat from core client for 30 sec - exiting 06:55:39 (572): No heartbeat from core client for 30 sec - exiting 06:55:40 (572): No heartbeat from core client for 30 sec - exiting 06:55:41 (572): No heartbeat from core client for 30 sec - exiting 06:55:42 (572): No heartbeat from core client for 30 sec - exiting 06:55:43 (572): No heartbeat from core client for 30 sec - exiting 06:55:44 (572): No heartbeat from core client for 30 sec - exiting 06:55:45 (572): No heartbeat from core client for 30 sec - exiting 06:55:46 (572): No heartbeat from core client for 30 sec - exiting 06:55:48 (572): No heartbeat from core client for 30 sec - exiting 06:55:49 (572): No heartbeat from core client for 30 sec - exiting 06:55:50 (572): No heartbeat from core client for 30 sec - exiting 06:55:51 (572): No heartbeat from core client for 30 sec - exiting 06:55:52 (572): No heartbeat from core client for 30 sec - exiting 06:55:53 (572): No heartbeat from core client for 30 sec - exiting 06:55:54 (572): No heartbeat from core client for 30 sec - exiting 06:55:55 (572): No heartbeat from core client for 30 sec - exiting 06:55:56 (572): No heartbeat from core client for 30 sec - exiting 06:55:57 (572): No heartbeat from core client for 30 sec - exiting 06:55:58 (572): No heartbeat from core client for 30 sec - exiting 06:56:00 (572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:56:01 (572): No heartbeat from core client for 30 sec - exiting 06:56:02 (572): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 06:59:01 (412): No heartbeat from core client for 30 sec - exiting 06:59:02 (412): No heartbeat from core client for 30 sec - exiting 06:59:03 (412): No heartbeat from core client for 30 sec - exiting 06:59:04 (412): No heartbeat from core client for 30 sec - exiting 06:59:06 (412): No heartbeat from core client for 30 sec - exiting 06:59:07 (412): No heartbeat from core client for 30 sec - exiting 06:59:08 (412): No heartbeat from core client for 30 sec - exiting 06:59:09 (412): No heartbeat from core client for 30 sec - exiting 06:59:10 (412): No heartbeat from core client for 30 sec - exiting 06:59:11 (412): No heartbeat from core client for 30 sec - exiting 06:59:12 (412): No heartbeat from core client for 30 sec - exiting 06:59:13 (412): No heartbeat from core client for 30 sec - exiting 06:59:14 (412): No heartbeat from core client for 30 sec - exiting 06:59:15 (412): No heartbeat from core client for 30 sec - exiting 06:59:17 (412): No heartbeat from core client for 30 sec - exiting 06:59:18 (412): No heartbeat from core client for 30 sec - exiting 06:59:19 (412): No heartbeat from core client for 30 sec - exiting 06:59:20 (412): No heartbeat from core client for 30 sec - exiting 06:59:21 (412): No heartbeat from core client for 30 sec - exiting 06:59:22 (412): No heartbeat from core client for 30 sec - exiting 06:59:23 (412): No heartbeat from core client for 30 sec - exiting 06:59:24 (412): No heartbeat from core client for 30 sec - exiting 06:59:25 (412): No heartbeat from core client for 30 sec - exiting 06:59:26 (412): No heartbeat from core client for 30 sec - exiting 06:59:27 (412): No heartbeat from core client for 30 sec - exiting 06:59:29 (412): No heartbeat from core client for 30 sec - exiting 06:59:30 (412): No heartbeat from core client for 30 sec - exiting 06:59:31 (412): No heartbeat from core client for 30 sec - exiting 06:59:32 (412): No heartbeat from core client for 30 sec - exiting 06:59:33 (412): No heartbeat from core client for 30 sec - exiting 06:59:34 (412): No heartbeat from core client for 30 sec - exiting 06:59:35 (412): No heartbeat from core client for 30 sec - exiting 06:59:36 (412): No heartbeat from core client for 30 sec - exiting 06:59:37 (412): No heartbeat from core client for 30 sec - exiting 06:59:38 (412): No heartbeat from core client for 30 sec - exiting 06:59:39 (412): No heartbeat from core client for 30 sec - exiting 06:59:41 (412): No heartbeat from core client for 30 sec - exiting 06:59:42 (412): No heartbeat from core client for 30 sec - exiting 06:59:43 (412): No heartbeat from core client for 30 sec - exiting 06:59:44 (412): No heartbeat from core client for 30 sec - exiting 06:59:45 (412): No heartbeat from core client for 30 sec - exiting 06:59:46 (412): No heartbeat from core client for 30 sec - exiting 06:59:47 (412): No heartbeat from core client for 30 sec - exiting 06:59:48 (412): No heartbeat from core client for 30 sec - exiting 06:59:49 (412): No heartbeat from core client for 30 sec - exiting 06:59:50 (412): No heartbeat from core client for 30 sec - exiting 06:59:52 (412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:59:53 (412): No heartbeat from core client for 30 sec - exiting 06:59:54 (412): No heartbeat from core client for 30 sec - exiting 06:59:55 (412): No heartbeat from core client for 30 sec - exiting 06:59:56 (412): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=3956, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3996, selfPID=3904, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nac7_2012_1_008601015_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Aug 2014 04:02:21	1205870	16418329	hadam3p_anz_nac7_2012_1_008601015_0	46,379	296,995	6.4037
08 Aug 2014 05:43:18	1205870	16418329	hadam3p_anz_nac7_2012_1_008601015_0	34,859	212,201	6.0874
29 Apr 2014 07:20:47	1205870	16418329	hadam3p_anz_nac7_2012_1_008601015_0	23,339	133,898	5.7371
15 Apr 2014 07:18:45	1205870	16418329	hadam3p_anz_nac7_2012_1_008601015_0	11,819	71,158	6.0206