Task 12888173

Name	hadam3p_saf_1dom_1981_1_006945566_1
Workunit	7148882
Created	16 May 2011, 18:40:58 UTC
Sent	16 May 2011, 18:41:04 UTC
Report deadline	28 Apr 2012, 0:01:04 UTC
Received	5 Jun 2011, 4:55:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1087039
Run time	14 hours 7 min 32 sec
CPU time	12 hours 28 min 26 sec
Validate state	Invalid
Credit	188.44
Device peak FLOPS	2.63 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3868, selfPID=3868, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2928, selfPID=2928, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5360, selfPID=5360, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:31:58 (3676): No heartbeat from core client for 30 sec - exiting 00:31:59 (3676): No heartbeat from core client for 30 sec - exiting 00:32:00 (3676): No heartbeat from core client for 30 sec - exiting 00:32:02 (3676): No heartbeat from core client for 30 sec - exiting 00:32:03 (3676): No heartbeat from core client for 30 sec - exiting 00:32:04 (3676): No heartbeat from core client for 30 sec - exiting 00:32:05 (3676): No heartbeat from core client for 30 sec - exiting 00:32:06 (3676): No heartbeat from core client for 30 sec - exiting 00:32:07 (3676): No heartbeat from core client for 30 sec - exiting 00:32:08 (3676): No heartbeat from core client for 30 sec - exiting 00:32:09 (3676): No heartbeat from core client for 30 sec - exiting 00:32:10 (3676): No heartbeat from core client for 30 sec - exiting 00:32:11 (3676): No heartbeat from core client for 30 sec - exiting 00:32:12 (3676): No heartbeat from core client for 30 sec - exiting 00:32:14 (3676): No heartbeat from core client for 30 sec - exiting 00:32:15 (3676): No heartbeat from core client for 30 sec - exiting 00:32:16 (3676): No heartbeat from core client for 30 sec - exiting 00:32:17 (3676): No heartbeat from core client for 30 sec - exiting 00:32:18 (3676): No heartbeat from core client for 30 sec - exiting 00:32:19 (3676): No heartbeat from core client for 30 sec - exiting 00:32:20 (3676): No heartbeat from core client for 30 sec - exiting 00:32:21 (3676): No heartbeat from core client for 30 sec - exiting 00:32:22 (3676): No heartbeat from core client for 30 sec - exiting 00:32:23 (3676): No heartbeat from core client for 30 sec - exiting 00:32:25 (3676): No heartbeat from core client for 30 sec - exiting 00:32:26 (3676): No heartbeat from core client for 30 sec - exiting 00:32:27 (3676): No heartbeat from core client for 30 sec - exiting 00:32:28 (3676): No heartbeat from core client for 30 sec - exiting 00:32:29 (3676): No heartbeat from core client for 30 sec - exiting 00:32:30 (3676): No heartbeat from core client for 30 sec - exiting 00:32:31 (3676): No heartbeat from core client for 30 sec - exiting 00:32:32 (3676): No heartbeat from core client for 30 sec - exiting 00:32:33 (3676): No heartbeat from core client for 30 sec - exiting 00:32:34 (3676): No heartbeat from core client for 30 sec - exiting 00:32:35 (3676): No heartbeat from core client for 30 sec - exiting 00:32:37 (3676): No heartbeat from core client for 30 sec - exiting 00:32:38 (3676): No heartbeat from core client for 30 sec - exiting 00:32:39 (3676): No heartbeat from core client for 30 sec - exiting 00:32:40 (3676): No heartbeat from core client for 30 sec - exiting 00:32:41 (3676): No heartbeat from core client for 30 sec - exiting 00:32:42 (3676): No heartbeat from core client for 30 sec - exiting 00:32:43 (3676): No heartbeat from core client for 30 sec - exiting 00:32:44 (3676): No heartbeat from core client for 30 sec - exiting 00:32:45 (3676): No heartbeat from core client for 30 sec - exiting 00:32:46 (3676): No heartbeat from core client for 30 sec - exiting 00:32:48 (3676): No heartbeat from core client for 30 sec - exiting 00:32:49 (3676): No heartbeat from core client for 30 sec - exiting 00:32:50 (3676): No heartbeat from core client for 30 sec - exiting 00:32:51 (3676): No heartbeat from core client for 30 sec - exiting 00:32:52 (3676): No heartbeat from core client for 30 sec - exiting 00:32:53 (3676): No heartbeat from core client for 30 sec - exiting 00:32:54 (3676): No heartbeat from core client for 30 sec - exiting 00:32:55 (3676): No heartbeat from core client for 30 sec - exiting 00:32:56 (3676): No heartbeat from core client for 30 sec - exiting 00:32:57 (3676): No heartbeat from core client for 30 sec - exiting 00:32:58 (3676): No heartbeat from core client for 30 sec - exiting 00:33:00 (3676): No heartbeat from core client for 30 sec - exiting 00:33:01 (3676): No heartbeat from core client for 30 sec - exiting 00:33:02 (3676): No heartbeat from core client for 30 sec - exiting 00:33:03 (3676): No heartbeat from core client for 30 sec - exiting 00:33:04 (3676): No heartbeat from core client for 30 sec - exiting 00:33:05 (3676): No heartbeat from core client for 30 sec - exiting 00:33:06 (3676): No heartbeat from core client for 30 sec - exiting 00:33:07 (3676): No heartbeat from core client for 30 sec - exiting 00:33:08 (3676): No heartbeat from core client for 30 sec - exiting 00:33:09 (3676): No heartbeat from core client for 30 sec - exiting 00:33:10 (3676): No heartbeat from core client for 30 sec - exiting 00:33:12 (3676): No heartbeat from core client for 30 sec - exiting 00:33:13 (3676): No heartbeat from core client for 30 sec - exiting 00:33:14 (3676): No heartbeat from core client for 30 sec - exiting 00:33:15 (3676): No heartbeat from core client for 30 sec - exiting 00:33:16 (3676): No heartbeat from core client for 30 sec - exiting 00:33:17 (3676): No heartbeat from core client for 30 sec - exiting 00:33:18 (3676): No heartbeat from core client for 30 sec - exiting 00:33:19 (3676): No heartbeat from core client for 30 sec - exiting 00:33:20 (3676): No heartbeat from core client for 30 sec - exiting 00:33:21 (3676): No heartbeat from core client for 30 sec - exiting 00:33:23 (3676): No heartbeat from core client for 30 sec - exiting 00:33:24 (3676): No heartbeat from core client for 30 sec - exiting 00:33:25 (3676): No heartbeat from core client for 30 sec - exiting 00:33:26 (3676): No heartbeat from core client for 30 sec - exiting 00:33:27 (3676): No heartbeat from core client for 30 sec - exiting 00:33:28 (3676): No heartbeat from core client for 30 sec - exiting 00:33:29 (3676): No heartbeat from core client for 30 sec - exiting 00:33:30 (3676): No heartbeat from core client for 30 sec - exiting 00:33:31 (3676): No heartbeat from core client for 30 sec - exiting 00:33:32 (3676): No heartbeat from core client for 30 sec - exiting 00:33:34 (3676): No heartbeat from core client for 30 sec - exiting 00:33:35 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:35:42 (3096): No heartbeat from core client for 30 sec - exiting 01:35:43 (3096): No heartbeat from core client for 30 sec - exiting 01:35:44 (3096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3852, selfPID=3852, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5508, selfPID=5508, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5604, selfPID=5604, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3756, selfPID=3756, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1780, selfPID=1780, iMonCtr=2 05:05:19 (4544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:07:17 (4544): No heartbeat from core client for 30 sec - exiting 05:18:34 (5148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:19:11 (5148): No heartbeat from core client for 30 sec - exiting 05:21:07 (608): No heartbeat from core client for 30 sec - exiting 05:21:08 (608): No heartbeat from core client for 30 sec - exiting 05:21:09 (608): No heartbeat from core client for 30 sec - exiting 05:21:10 (608): No heartbeat from core client for 30 sec - exiting 05:21:11 (608): No heartbeat from core client for 30 sec - exiting 05:21:12 (608): No heartbeat from core client for 30 sec - exiting 05:21:13 (608): No heartbeat from core client for 30 sec - exiting 05:21:14 (608): No heartbeat from core client for 30 sec - exiting 05:21:16 (608): No heartbeat from core client for 30 sec - exiting 05:21:17 (608): No heartbeat from core client for 30 sec - exiting 05:21:18 (608): No heartbeat from core client for 30 sec - exiting 05:21:19 (608): No heartbeat from core client for 30 sec - exiting 05:21:20 (608): No heartbeat from core client for 30 sec - exiting 05:21:21 (608): No heartbeat from core client for 30 sec - exiting 05:21:22 (608): No heartbeat from core client for 30 sec - exiting 05:21:23 (608): No heartbeat from core client for 30 sec - exiting 05:21:24 (608): No heartbeat from core client for 30 sec - exiting 05:21:25 (608): No heartbeat from core client for 30 sec - exiting 05:21:26 (608): No heartbeat from core client for 30 sec - exiting 05:21:28 (608): No heartbeat from core client for 30 sec - exiting 05:21:29 (608): No heartbeat from core client for 30 sec - exiting 05:21:30 (608): No heartbeat from core client for 30 sec - exiting 05:21:31 (608): No heartbeat from core client for 30 sec - exiting 05:21:32 (608): No heartbeat from core client for 30 sec - exiting 05:21:33 (608): No heartbeat from core client for 30 sec - exiting 05:21:34 (608): No heartbeat from core client for 30 sec - exiting 05:21:35 (608): No heartbeat from core client for 30 sec - exiting 05:21:36 (608): No heartbeat from core client for 30 sec - exiting 05:21:37 (608): No heartbeat from core client for 30 sec - exiting 05:21:39 (608): No heartbeat from core client for 30 sec - exiting 05:21:40 (608): No heartbeat from core client for 30 sec - exiting 05:21:41 (608): No heartbeat from core client for 30 sec - exiting 05:21:42 (608): No heartbeat from core client for 30 sec - exiting 05:21:43 (608): No heartbeat from core client for 30 sec - exiting 05:21:44 (608): No heartbeat from core client for 30 sec - exiting 05:21:45 (608): No heartbeat from core client for 30 sec - exiting 05:21:46 (608): No heartbeat from core client for 30 sec - exiting 05:21:47 (608): No heartbeat from core client for 30 sec - exiting 05:21:48 (608): No heartbeat from core client for 30 sec - exiting 05:21:49 (608): No heartbeat from core client for 30 sec - exiting 05:21:51 (608): No heartbeat from core client for 30 sec - exiting 05:21:52 (608): No heartbeat from core client for 30 sec - exiting 05:21:53 (608): No heartbeat from core client for 30 sec - exiting 05:21:54 (608): No heartbeat from core client for 30 sec - exiting 05:21:55 (608): No heartbeat from core client for 30 sec - exiting 05:21:56 (608): No heartbeat from core client for 30 sec - exiting 05:21:57 (608): No heartbeat from core client for 30 sec - exiting 05:21:58 (608): No heartbeat from core client for 30 sec - exiting 05:21:59 (608): No heartbeat from core client for 30 sec - exiting 05:22:00 (608): No heartbeat from core client for 30 sec - exiting 05:22:01 (608): No heartbeat from core client for 30 sec - exiting 05:22:03 (608): No heartbeat from core client for 30 sec - exiting 05:22:04 (608): No heartbeat from core client for 30 sec - exiting 05:22:05 (608): No heartbeat from core client for 30 sec - exiting 05:22:06 (608): No heartbeat from core client for 30 sec - exiting 05:22:07 (608): No heartbeat from core client for 30 sec - exiting 05:22:08 (608): No heartbeat from core client for 30 sec - exiting 05:22:09 (608): No heartbeat from core client for 30 sec - exiting 05:22:10 (608): No heartbeat from core client for 30 sec - exiting 05:22:11 (608): No heartbeat from core client for 30 sec - exiting 05:22:12 (608): No heartbeat from core client for 30 sec - exiting 05:22:14 (608): No heartbeat from core client for 30 sec - exiting 05:22:15 (608): No heartbeat from core client for 30 sec - exiting 05:22:16 (608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:43 (1828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:31 (4756): No heartbeat from core client for 30 sec - exiting 05:34:32 (4756): No heartbeat from core client for 30 sec - exiting 05:34:33 (4756): No heartbeat from core client for 30 sec - exiting 05:34:34 (4756): No heartbeat from core client for 30 sec - exiting 05:34:36 (4756): No heartbeat from core client for 30 sec - exiting 05:34:37 (4756): No heartbeat from core client for 30 sec - exiting 05:34:38 (4756): No heartbeat from core client for 30 sec - exiting 05:34:39 (4756): No heartbeat from core client for 30 sec - exiting 05:34:40 (4756): No heartbeat from core client for 30 sec - exiting 05:34:41 (4756): No heartbeat from core client for 30 sec - exiting 05:34:42 (4756): No heartbeat from core client for 30 sec - exiting 05:34:43 (4756): No heartbeat from core client for 30 sec - exiting 05:34:44 (4756): No heartbeat from core client for 30 sec - exiting 05:34:45 (4756): No heartbeat from core client for 30 sec - exiting 05:34:46 (4756): No heartbeat from core client for 30 sec - exiting 05:34:48 (4756): No heartbeat from core client for 30 sec - exiting 05:34:49 (4756): No heartbeat from core client for 30 sec - exiting 05:34:50 (4756): No heartbeat from core client for 30 sec - exiting 05:34:51 (4756): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=4576, iMonCtr=1 05:35:36 (4756): No heartbeat from core client for 30 sec - exiting 05:35:37 (4756): No heartbeat from core client for 30 sec - exiting 05:35:38 (4756): No heartbeat from core client for 30 sec - exiting 05:35:39 (4756): No heartbeat from core client for 30 sec - exiting 05:35:40 (4756): No heartbeat from core client for 30 sec - exiting 05:35:41 (4756): No heartbeat from core client for 30 sec - exiting 05:35:42 (4756): No heartbeat from core client for 30 sec - exiting 05:35:43 (4756): No heartbeat from core client for 30 sec - exiting 05:35:44 (4756): No heartbeat from core client for 30 sec - exiting 05:35:46 (4756): No heartbeat from core client for 30 sec - exiting 05:35:47 (4756): No heartbeat from core client for 30 sec - exiting 05:35:48 (4756): No heartbeat from core client for 30 sec - exiting 05:35:49 (4756): No heartbeat from core client for 30 sec - exiting 05:35:50 (4756): No heartbeat from core client for 30 sec - exiting 05:35:51 (4756): No heartbeat from core client for 30 sec - exiting 05:35:52 (4756): No heartbeat from core client for 30 sec - exiting 05:35:53 (4756): No heartbeat from core client for 30 sec - exiting 05:35:54 (4756): No heartbeat from core client for 30 sec - exiting 05:35:55 (4756): No heartbeat from core client for 30 sec - exiting 05:35:56 (4756): No heartbeat from core client for 30 sec - exiting 05:35:58 (4756): No heartbeat from core client for 30 sec - exiting 05:35:59 (4756): No heartbeat from core client for 30 sec - exiting 05:36:00 (4756): No heartbeat from core client for 30 sec - exiting 05:36:01 (4756): No heartbeat from core client for 30 sec - exiting 05:36:02 (4756): No heartbeat from core client for 30 sec - exiting 05:36:03 (4756): No heartbeat from core client for 30 sec - exiting 05:36:04 (4756): No heartbeat from core client for 30 sec - exiting 05:36:05 (4756): No heartbeat from core client for 30 sec - exiting 05:36:06 (4756): No heartbeat from core client for 30 sec - exiting 05:36:07 (4756): No heartbeat from core client for 30 sec - exiting 05:36:09 (4756): No heartbeat from core client for 30 sec - exiting 05:36:10 (4756): No heartbeat from core client for 30 sec - exiting 05:36:11 (4756): No heartbeat from core client for 30 sec - exiting 05:36:12 (4756): No heartbeat from core client for 30 sec - exiting 05:36:13 (4756): No heartbeat from core client for 30 sec - exiting 05:36:14 (4756): No heartbeat from core client for 30 sec - exiting 05:36:15 (4756): No heartbeat from core client for 30 sec - exiting 05:36:16 (4756): No heartbeat from core client for 30 sec - exiting 05:36:17 (4756): No heartbeat from core client for 30 sec - exiting 05:36:18 (4756): No heartbeat from core client for 30 sec - exiting 05:36:19 (4756): No heartbeat from core client for 30 sec - exiting 05:36:21 (4756): No heartbeat from core client for 30 sec - exiting 05:36:22 (4756): No heartbeat from core client for 30 sec - exiting 05:36:23 (4756): No heartbeat from core client for 30 sec - exiting 05:36:24 (4756): No heartbeat from core client for 30 sec - exiting 05:36:25 (4756): No heartbeat from core client for 30 sec - exiting 05:36:26 (4756): No heartbeat from core client for 30 sec - exiting 05:36:27 (4756): No heartbeat from core client for 30 sec - exiting 05:36:28 (3148): Can't acquire lockfile (32) - waiting 35s Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=1552, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=948, selfPID=3148, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 05:40:22 (3148): No heartbeat from core client for 30 sec - exiting 05:40:54 (3148): No heartbeat from core client for 30 sec - exiting 05:41:27 (3148): No heartbeat from core client for 30 sec - exiting Called boinc_finish 05:41:44 (3148): No heartbeat from core client for 30 sec - exiting 05:41:45 (3148): No heartbeat from core client for 30 sec - exiting </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_2.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_3.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_4.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_1dom_1981_1_006945566_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 May 2011 21:30:09	1087039	12888173	hadam3p_saf_1dom_1981_1_006945566_1	11,616	29,842	2.5690