Task 16767790

Name	hadam3p_eu_e5dm_2013_1_008848757_0
Workunit	8994686
Created	8 Jul 2014, 19:05:06 UTC
Sent	20 Jul 2014, 5:26:00 UTC
Report deadline	2 Jul 2015, 10:46:00 UTC
Received	28 Jul 2014, 18:51:04 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1329821
Run time	3 days 19 hours 16 min 52 sec
CPU time	3 days 18 hours 32 min 16 sec
Validate state	Invalid
Credit	796.57
Device peak FLOPS	1.77 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:02:18 (2196): No heartbeat from core client for 30 sec - exiting 19:02:19 (2196): No heartbeat from core client for 30 sec - exiting 19:02:20 (2196): No heartbeat from core client for 30 sec - exiting 19:02:21 (2196): No heartbeat from core client for 30 sec - exiting 19:02:22 (2196): No heartbeat from core client for 30 sec - exiting 19:02:23 (2196): No heartbeat from core client for 30 sec - exiting 19:02:24 (2196): No heartbeat from core client for 30 sec - exiting 19:02:25 (2196): No heartbeat from core client for 30 sec - exiting 19:02:26 (2196): No heartbeat from core client for 30 sec - exiting 19:02:27 (2196): No heartbeat from core client for 30 sec - exiting 19:02:28 (2196): No heartbeat from core client for 30 sec - exiting 19:02:29 (2196): No heartbeat from core client for 30 sec - exiting 19:02:30 (2196): No heartbeat from core client for 30 sec - exiting 19:02:31 (2196): No heartbeat from core client for 30 sec - exiting 19:02:32 (2196): No heartbeat from core client for 30 sec - exiting 19:02:33 (2196): No heartbeat from core client for 30 sec - exiting 19:02:34 (2196): No heartbeat from core client for 30 sec - exiting 19:02:35 (2196): No heartbeat from core client for 30 sec - exiting 19:02:36 (2196): No heartbeat from core client for 30 sec - exiting 19:02:37 (2196): No heartbeat from core client for 30 sec - exiting 19:02:38 (2196): No heartbeat from core client for 30 sec - exiting 19:02:39 (2196): No heartbeat from core client for 30 sec - exiting 19:02:40 (2196): No heartbeat from core client for 30 sec - exiting 19:02:41 (2196): No heartbeat from core client for 30 sec - exiting 19:02:42 (2196): No heartbeat from core client for 30 sec - exiting 19:02:43 (2196): No heartbeat from core client for 30 sec - exiting 19:02:44 (2196): No heartbeat from core client for 30 sec - exiting 19:02:45 (2196): No heartbeat from core client for 30 sec - exiting 19:02:46 (2196): No heartbeat from core client for 30 sec - exiting 19:02:47 (2196): No heartbeat from core client for 30 sec - exiting 19:02:48 (2196): No heartbeat from core client for 30 sec - exiting 19:02:49 (2196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:02:06 (2166): No heartbeat from core client for 30 sec - exiting 19:02:07 (2166): No heartbeat from core client for 30 sec - exiting 19:02:08 (2166): No heartbeat from core client for 30 sec - exiting 19:02:09 (2166): No heartbeat from core client for 30 sec - exiting 19:02:10 (2166): No heartbeat from core client for 30 sec - exiting 19:02:11 (2166): No heartbeat from core client for 30 sec - exiting 19:02:12 (2166): No heartbeat from core client for 30 sec - exiting 19:02:13 (2166): No heartbeat from core client for 30 sec - exiting 19:02:14 (2166): No heartbeat from core client for 30 sec - exiting 19:02:15 (2166): No heartbeat from core client for 30 sec - exiting 19:02:16 (2166): No heartbeat from core client for 30 sec - exiting 19:02:17 (2166): No heartbeat from core client for 30 sec - exiting 19:02:18 (2166): No heartbeat from core client for 30 sec - exiting 19:02:19 (2166): No heartbeat from core client for 30 sec - exiting 19:02:20 (2166): No heartbeat from core client for 30 sec - exiting 19:02:21 (2166): No heartbeat from core client for 30 sec - exiting 19:02:22 (2166): No heartbeat from core client for 30 sec - exiting 19:02:23 (2166): No heartbeat from core client for 30 sec - exiting 19:02:24 (2166): No heartbeat from core client for 30 sec - exiting 19:02:25 (2166): No heartbeat from core client for 30 sec - exiting 19:02:26 (2166): No heartbeat from core client for 30 sec - exiting 19:02:27 (2166): No heartbeat from core client for 30 sec - exiting 19:02:28 (2166): No heartbeat from core client for 30 sec - exiting 19:02:29 (2166): No heartbeat from core client for 30 sec - exiting 19:02:30 (2166): No heartbeat from core client for 30 sec - exiting 19:02:31 (2166): No heartbeat from core client for 30 sec - exiting 19:02:32 (2166): No heartbeat from core client for 30 sec - exiting 19:02:33 (2166): No heartbeat from core client for 30 sec - exiting 19:02:34 (2166): No heartbeat from core client for 30 sec - exiting 19:02:35 (2166): No heartbeat from core client for 30 sec - exiting 19:02:36 (2166): No heartbeat from core client for 30 sec - exiting 19:02:37 (2166): No heartbeat from core client for 30 sec - exiting 19:02:38 (2166): No heartbeat from core client for 30 sec - exiting 19:02:39 (2166): No heartbeat from core client for 30 sec - exiting 19:02:40 (2166): No heartbeat from core client for 30 sec - exiting 19:02:41 (2166): No heartbeat from core client for 30 sec - exiting 19:02:42 (2166): No heartbeat from core client for 30 sec - exiting 19:02:43 (2166): No heartbeat from core client for 30 sec - exiting 19:02:44 (2166): No heartbeat from core client for 30 sec - exiting 19:02:45 (2166): No heartbeat from core client for 30 sec - exiting 19:02:46 (2166): No heartbeat from core client for 30 sec - exiting 19:02:47 (2166): No heartbeat from core client for 30 sec - exiting 19:02:48 (2166): No heartbeat from core client for 30 sec - exiting 19:02:49 (2166): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:19:08 (2002): No heartbeat from core client for 30 sec - exiting 01:19:09 (2002): No heartbeat from core client for 30 sec - exiting 01:19:10 (2002): No heartbeat from core client for 30 sec - exiting 01:19:11 (2002): No heartbeat from core client for 30 sec - exiting 01:19:12 (2002): No heartbeat from core client for 30 sec - exiting 01:19:13 (2002): No heartbeat from core client for 30 sec - exiting 01:19:14 (2002): No heartbeat from core client for 30 sec - exiting 01:19:15 (2002): No heartbeat from core client for 30 sec - exiting 01:19:16 (2002): No heartbeat from core client for 30 sec - exiting 01:19:17 (2002): No heartbeat from core client for 30 sec - exiting 01:19:18 (2002): No heartbeat from core client for 30 sec - exiting 01:19:19 (2002): No heartbeat from core client for 30 sec - exiting 01:19:20 (2002): No heartbeat from core client for 30 sec - exiting 01:19:21 (2002): No heartbeat from core client for 30 sec - exiting 01:19:22 (2002): No heartbeat from core client for 30 sec - exiting 01:19:26 (2002): No heartbeat from core client for 30 sec - exiting 01:19:27 (2002): No heartbeat from core client for 30 sec - exiting 01:19:28 (2002): No heartbeat from core client for 30 sec - exiting 01:19:30 (2002): No heartbeat from core client for 30 sec - exiting 01:19:31 (2002): No heartbeat from core client for 30 sec - exiting 01:19:32 (2002): No heartbeat from core client for 30 sec - exiting 01:19:33 (2002): No heartbeat from core client for 30 sec - exiting 01:19:34 (2002): No heartbeat from core client for 30 sec - exiting 01:19:35 (2002): No heartbeat from core client for 30 sec - exiting 01:19:36 (2002): No heartbeat from core client for 30 sec - exiting 01:19:37 (2002): No heartbeat from core client for 30 sec - exiting 01:19:38 (2002): No heartbeat from core client for 30 sec - exiting 01:19:39 (2002): No heartbeat from core client for 30 sec - exiting 01:19:40 (2002): No heartbeat from core client for 30 sec - exiting 01:19:41 (2002): No heartbeat from core client for 30 sec - exiting 01:19:42 (2002): No heartbeat from core client for 30 sec - exiting 01:19:43 (2002): No heartbeat from core client for 30 sec - exiting 01:19:44 (2002): No heartbeat from core client for 30 sec - exiting 01:19:45 (2002): No heartbeat from core client for 30 sec - exiting 01:19:46 (2002): No heartbeat from core client for 30 sec - exiting 01:19:47 (2002): No heartbeat from core client for 30 sec - exiting 01:19:48 (2002): No heartbeat from core client for 30 sec - exiting 01:19:49 (2002): No heartbeat from core client for 30 sec - exiting 01:19:50 (2002): No heartbeat from core client for 30 sec - exiting 01:19:51 (2002): No heartbeat from core client for 30 sec - exiting 01:19:52 (2002): No heartbeat from core client for 30 sec - exiting 01:19:53 (2002): No heartbeat from core client for 30 sec - exiting 01:19:54 (2002): No heartbeat from core client for 30 sec - exiting 01:19:55 (2002): No heartbeat from core client for 30 sec - exiting 01:19:56 (2002): No heartbeat from core client for 30 sec - exiting 01:19:57 (2002): No heartbeat from core client for 30 sec - exiting 01:19:58 (2002): No heartbeat from core client for 30 sec - exiting 01:19:59 (2002): No heartbeat from core client for 30 sec - exiting 01:20:00 (2002): No heartbeat from core client for 30 sec - exiting 01:20:01 (2002): No heartbeat from core client for 30 sec - exiting 01:20:02 (2002): No heartbeat from core client for 30 sec - exiting 01:20:03 (2002): No heartbeat from core client for 30 sec - exiting 01:20:04 (2002): No heartbeat from core client for 30 sec - exiting 01:20:05 (2002): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:02:04 (2209): No heartbeat from core client for 30 sec - exiting 19:02:05 (2209): No heartbeat from core client for 30 sec - exiting 19:02:06 (2209): No heartbeat from core client for 30 sec - exiting 19:02:07 (2209): No heartbeat from core client for 30 sec - exiting 19:02:08 (2209): No heartbeat from core client for 30 sec - exiting 19:02:09 (2209): No heartbeat from core client for 30 sec - exiting 19:02:10 (2209): No heartbeat from core client for 30 sec - exiting 19:02:11 (2209): No heartbeat from core client for 30 sec - exiting 19:02:12 (2209): No heartbeat from core client for 30 sec - exiting 19:02:13 (2209): No heartbeat from core client for 30 sec - exiting 19:02:14 (2209): No heartbeat from core client for 30 sec - exiting 19:02:15 (2209): No heartbeat from core client for 30 sec - exiting 19:02:16 (2209): No heartbeat from core client for 30 sec - exiting 19:02:17 (2209): No heartbeat from core client for 30 sec - exiting 19:02:18 (2209): No heartbeat from core client for 30 sec - exiting 19:02:19 (2209): No heartbeat from core client for 30 sec - exiting 19:02:20 (2209): No heartbeat from core client for 30 sec - exiting 19:02:21 (2209): No heartbeat from core client for 30 sec - exiting 19:02:22 (2209): No heartbeat from core client for 30 sec - exiting 19:02:23 (2209): No heartbeat from core client for 30 sec - exiting 19:02:24 (2209): No heartbeat from core client for 30 sec - exiting 19:02:25 (2209): No heartbeat from core client for 30 sec - exiting 19:02:26 (2209): No heartbeat from core client for 30 sec - exiting 19:02:27 (2209): No heartbeat from core client for 30 sec - exiting 19:02:28 (2209): No heartbeat from core client for 30 sec - exiting 19:02:29 (2209): No heartbeat from core client for 30 sec - exiting 19:02:32 (2209): No heartbeat from core client for 30 sec - exiting 19:02:33 (2209): No heartbeat from core client for 30 sec - exiting 19:02:34 (2209): No heartbeat from core client for 30 sec - exiting 19:02:35 (2209): No heartbeat from core client for 30 sec - exiting 19:02:36 (2209): No heartbeat from core client for 30 sec - exiting 19:02:37 (2209): No heartbeat from core client for 30 sec - exiting 19:02:38 (2209): No heartbeat from core client for 30 sec - exiting 19:02:39 (2209): No heartbeat from core client for 30 sec - exiting 19:02:40 (2209): No heartbeat from core client for 30 sec - exiting 19:02:41 (2209): No heartbeat from core client for 30 sec - exiting 19:02:42 (2209): No heartbeat from core client for 30 sec - exiting 19:02:43 (2209): No heartbeat from core client for 30 sec - exiting 19:02:44 (2209): No heartbeat from core client for 30 sec - exiting 19:02:45 (2209): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:02:03 (2182): No heartbeat from core client for 30 sec - exiting 19:02:04 (2182): No heartbeat from core client for 30 sec - exiting 19:02:05 (2182): No heartbeat from core client for 30 sec - exiting 19:02:06 (2182): No heartbeat from core client for 30 sec - exiting 19:02:07 (2182): No heartbeat from core client for 30 sec - exiting 19:02:08 (2182): No heartbeat from core client for 30 sec - exiting 19:02:09 (2182): No heartbeat from core client for 30 sec - exiting 19:02:10 (2182): No heartbeat from core client for 30 sec - exiting 19:02:11 (2182): No heartbeat from core client for 30 sec - exiting 19:02:12 (2182): No heartbeat from core client for 30 sec - exiting 19:02:13 (2182): No heartbeat from core client for 30 sec - exiting 19:02:14 (2182): No heartbeat from core client for 30 sec - exiting 19:02:15 (2182): No heartbeat from core client for 30 sec - exiting 19:02:16 (2182): No heartbeat from core client for 30 sec - exiting 19:02:17 (2182): No heartbeat from core client for 30 sec - exiting 19:02:18 (2182): No heartbeat from core client for 30 sec - exiting 19:02:19 (2182): No heartbeat from core client for 30 sec - exiting 19:02:20 (2182): No heartbeat from core client for 30 sec - exiting 19:02:21 (2182): No heartbeat from core client for 30 sec - exiting 19:02:22 (2182): No heartbeat from core client for 30 sec - exiting 19:02:23 (2182): No heartbeat from core client for 30 sec - exiting 19:02:24 (2182): No heartbeat from core client for 30 sec - exiting 19:02:25 (2182): No heartbeat from core client for 30 sec - exiting 19:02:26 (2182): No heartbeat from core client for 30 sec - exiting 19:02:27 (2182): No heartbeat from core client for 30 sec - exiting 19:02:28 (2182): No heartbeat from core client for 30 sec - exiting 19:02:29 (2182): No heartbeat from core client for 30 sec - exiting 19:02:30 (2182): No heartbeat from core client for 30 sec - exiting 19:02:31 (2182): No heartbeat from core client for 30 sec - exiting 19:02:32 (2182): No heartbeat from core client for 30 sec - exiting 19:02:33 (2182): No heartbeat from core client for 30 sec - exiting 19:02:34 (2182): No heartbeat from core client for 30 sec - exiting 19:02:35 (2182): No heartbeat from core client for 30 sec - exiting 19:02:36 (2182): No heartbeat from core client for 30 sec - exiting 19:02:37 (2182): No heartbeat from core client for 30 sec - exiting 19:02:38 (2182): No heartbeat from core client for 30 sec - exiting 19:02:39 (2182): No heartbeat from core client for 30 sec - exiting 19:02:40 (2182): No heartbeat from core client for 30 sec - exiting 19:02:41 (2182): No heartbeat from core client for 30 sec - exiting 19:02:42 (2182): No heartbeat from core client for 30 sec - exiting 19:02:43 (2182): No heartbeat from core client for 30 sec - exiting 19:02:45 (2182): No heartbeat from core client for 30 sec - exiting 19:02:46 (2182): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: READDM1A: Dump does not match STASH list - see output tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_e5dm_2013_1_008848757_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Jul 2014 05:35:52	1329821	16767790	hadam3p_eu_e5dm_2013_1_008848757_0	46,176	275,908	5.9751
25 Jul 2014 22:52:15	1329821	16767790	hadam3p_eu_e5dm_2013_1_008848757_0	34,656	206,714	5.9647
24 Jul 2014 04:11:30	1329821	16767790	hadam3p_eu_e5dm_2013_1_008848757_0	23,136	137,815	5.9567
22 Jul 2014 18:04:18	1329821	16767790	hadam3p_eu_e5dm_2013_1_008848757_0	11,616	69,496	5.9828