Task 12258363

Name	hadam3p_saf_1m2s_1963_1_006975644_0
Workunit	7178960
Created	23 Nov 2010, 14:38:53 UTC
Sent	26 Feb 2011, 17:10:09 UTC
Report deadline	8 Feb 2012, 22:30:09 UTC
Received	23 Jul 2011, 14:14:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1038022
Run time	3 days 5 hours 35 min 50 sec
CPU time	3 days 4 hours 21 min 41 sec
Validate state	Invalid
Credit	1,683.45
Device peak FLOPS	2.14 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.08 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:33:31 (7096): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5388, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:47:43 (5032): No heartbeat from core client for 30 sec - exiting 22:47:44 (5032): No heartbeat from core client for 30 sec - exiting 22:47:45 (5032): No heartbeat from core client for 30 sec - exiting 22:47:46 (5032): No heartbeat from core client for 30 sec - exiting 22:47:47 (5032): No heartbeat from core client for 30 sec - exiting 22:47:49 (5032): No heartbeat from core client for 30 sec - exiting 22:47:50 (5032): No heartbeat from core client for 30 sec - exiting 22:47:51 (5032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4432, selfPID=3956, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4432, selfPID=4432, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2860, selfPID=3992, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4388, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:20:25 (4852): No heartbeat from core client for 30 sec - exiting 21:20:26 (4852): No heartbeat from core client for 30 sec - exiting 21:20:27 (4852): No heartbeat from core client for 30 sec - exiting 21:20:28 (4852): No heartbeat from core client for 30 sec - exiting 21:20:29 (4852): No heartbeat from core client for 30 sec - exiting 21:20:30 (4852): No heartbeat from core client for 30 sec - exiting 21:20:31 (4852): No heartbeat from core client for 30 sec - exiting 21:20:32 (4852): No heartbeat from core client for 30 sec - exiting 21:20:33 (4852): No heartbeat from core client for 30 sec - exiting 21:20:34 (4852): No heartbeat from core client for 30 sec - exiting 21:20:35 (4852): No heartbeat from core client for 30 sec - exiting 21:20:37 (4852): No heartbeat from core client for 30 sec - exiting 21:20:38 (4852): No heartbeat from core client for 30 sec - exiting 21:20:39 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:40 (4852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 01:57:09 (2852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:12:54 (5308): No heartbeat from core client for 30 sec - exiting 08:12:56 (5308): No heartbeat from core client for 30 sec - exiting 08:12:57 (5308): No heartbeat from core client for 30 sec - exiting 08:12:58 (5308): No heartbeat from core client for 30 sec - exiting 08:12:59 (5308): No heartbeat from core client for 30 sec - exiting 08:13:00 (5308): No heartbeat from core client for 30 sec - exiting 08:13:01 (5308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:13:02 (5308): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:38:09 (6128): No heartbeat from core client for 30 sec - exiting 10:38:10 (6128): No heartbeat from core client for 30 sec - exiting 10:38:11 (6128): No heartbeat from core client for 30 sec - exiting 10:38:12 (6128): No heartbeat from core client for 30 sec - exiting 10:38:13 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:40 (5448): Can't acquire lockfile (32) - waiting 35s 10:39:00 (4380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3604, selfPID=3604, iMonCtr=2 10:40:26 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:41 (5684): No heartbeat from core client for 30 sec - exiting 10:54:42 (5684): No heartbeat from core client for 30 sec - exiting 10:54:43 (5684): No heartbeat from core client for 30 sec - exiting 10:54:44 (5684): No heartbeat from core client for 30 sec - exiting 10:54:45 (5684): No heartbeat from core client for 30 sec - exiting 10:54:47 (5684): No heartbeat from core client for 30 sec - exiting 10:54:48 (5684): No heartbeat from core client for 30 sec - exiting 10:54:49 (5684): No heartbeat from core client for 30 sec - exiting 10:54:50 (5684): No heartbeat from core client for 30 sec - exiting 10:54:51 (5684): No heartbeat from core client for 30 sec - exiting 10:54:52 (5684): No heartbeat from core client for 30 sec - exiting 10:54:53 (5684): No heartbeat from core client for 30 sec - exiting 10:54:54 (5684): No heartbeat from core client for 30 sec - exiting 10:54:55 (5684): No heartbeat from core client for 30 sec - exiting 10:54:56 (5684): No heartbeat from core client for 30 sec - exiting 10:54:57 (5684): No heartbeat from core client for 30 sec - exiting 10:54:59 (5684): No heartbeat from core client for 30 sec - exiting 10:55:00 (5684): No heartbeat from core client for 30 sec - exiting 10:55:01 (5684): No heartbeat from core client for 30 sec - exiting 10:55:02 (5684): No heartbeat from core client for 30 sec - exiting 10:55:03 (5684): No heartbeat from core client for 30 sec - exiting 10:55:04 (5684): No heartbeat from core client for 30 sec - exiting 10:55:05 (5684): No heartbeat from core client for 30 sec - exiting 10:55:06 (5684): No heartbeat from core client for 30 sec - exiting 10:55:07 (5684): No heartbeat from core client for 30 sec - exiting 10:55:08 (5684): No heartbeat from core client for 30 sec - exiting 10:55:09 (5684): No heartbeat from core client for 30 sec - exiting 10:55:11 (5684): No heartbeat from core client for 30 sec - exiting 10:55:12 (5684): No heartbeat from core client for 30 sec - exiting 10:55:13 (5684): No heartbeat from core client for 30 sec - exiting 10:55:14 (5684): No heartbeat from core client for 30 sec - exiting 10:55:15 (5684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... ReSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:51:50 (4620): No heartbeat from core client for 30 sec - exiting 14:51:51 (4620): No heartbeat from core client for 30 sec - exiting 14:51:52 (4620): No heartbeat from core client for 30 sec - exiting 14:51:53 (4620): No heartbeat from core client for 30 sec - exiting 14:51:54 (4620): No heartbeat from core client for 30 sec - exiting 14:51:56 (4620): No heartbeat from core client for 30 sec - exiting 14:51:57 (4620): No heartbeat from core client for 30 sec - exiting 14:51:58 (4620): No heartbeat from core client for 30 sec - exiting 14:51:59 (4620): No heartbeat from core client for 30 sec - exiting 14:52:00 (4620): No heartbeat from core client for 30 sec - exiting 14:52:01 (4620): No heartbeat from core client for 30 sec - exiting 14:52:02 (4620): No heartbeat from core client for 30 sec - exiting 14:52:03 (4620): No heartbeat from core client for 30 sec - exiting 14:52:04 (4620): No heartbeat from core client for 30 sec - exiting 14:52:05 (4620): No heartbeat from core client for 30 sec - exiting 14:52:06 (4620): No heartbeat from core client for 30 sec - exiting 14:52:08 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5476, selfPID=5476, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5600, selfPID=5448, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 01:11:39 (5448): called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_saf_1m2s_1963_1_006975644_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1m2s_1963_1_006975644_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1m2s_1963_1_006975644_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 19:32:57	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	103,776	253,398	2.4418
25 Jul 2011 18:21:26	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	92,256	226,202	2.4519
25 Jul 2011 18:03:27	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	80,736	198,822	2.4626
25 Jul 2011 15:43:11	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	69,216	171,095	2.4719
25 Jul 2011 13:36:20	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	57,696	142,973	2.4780
08 Jul 2011 17:51:45	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	46,176	114,616	2.4822
07 Jul 2011 15:36:29	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	34,656	85,737	2.4739
01 Jul 2011 23:29:45	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	23,136	57,245	2.4743
30 Jun 2011 23:44:32	1038022	12258363	hadam3p_saf_1m2s_1963_1_006975644_0	11,616	28,870	2.4854