Task 12284321

Name	hadam3p_pnw_znl0_1995_1_006999484_0
Workunit	7202800
Created	24 Nov 2010, 11:18:40 UTC
Sent	29 Jan 2011, 22:59:40 UTC
Report deadline	12 Jan 2012, 4:19:40 UTC
Received	20 Feb 2011, 18:15:36 UTC
Server state	Over
Outcome	Didn't need
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1075763
Run time	5 days 6 hours 20 min 29 sec
CPU time	4 days 17 hours 26 min 8 sec
Validate state	Invalid
Credit	2,755.56
Device peak FLOPS	2.58 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.08 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9936, selfPID=9936, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:51:38 (5704): No heartbeat from core client for 30 sec - exiting 19:51:39 (5704): No heartbeat from core client for 30 sec - exiting 19:51:40 (5704): No heartbeat from core client for 30 sec - exiting 19:51:41 (5704): No heartbeat from core client for 30 sec - exiting 19:51:42 (5704): No heartbeat from core client for 30 sec - exiting 19:51:43 (5704): No heartbeat from core client for 30 sec - exiting 19:51:44 (5704): No heartbeat from core client for 30 sec - exiting 19:51:45 (5704): No heartbeat from core client for 30 sec - exiting 19:51:46 (5704): No heartbeat from core client for 30 sec - exiting 19:51:47 (5704): No heartbeat from core client for 30 sec - exiting 19:51:48 (5704): No heartbeat from core client for 30 sec - exiting 19:51:49 (5704): No heartbeat from core client for 30 sec - exiting 19:51:50 (5704): No heartbeat from core client for 30 sec - exiting 19:51:51 (5704): No heartbeat from core client for 30 sec - exiting 19:51:52 (5704): No heartbeat from core client for 30 sec - exiting 19:51:53 (5704): No heartbeat from core client for 30 sec - exiting 19:51:54 (5704): No heartbeat from core client for 30 sec - exiting 19:51:55 (5704): No heartbeat from core client for 30 sec - exiting 19:51:56 (5704): No heartbeat from core client for 30 sec - exiting 19:51:57 (5704): No heartbeat from core client for 30 sec - exiting 19:51:58 (5704): No heartbeat from core client for 30 sec - exiting 19:51:59 (5704): No heartbeat from core client for 30 sec - exiting 19:52:00 (5704): No heartbeat from core client for 30 sec - exiting 19:52:01 (5704): No heartbeat from core client for 30 sec - exiting 19:52:02 (5704): No heartbeat from core client for 30 sec - exiting 19:52:03 (5704): No heartbeat from core client for 30 sec - exiting 19:52:04 (5704): No heartbeat from core client for 30 sec - exiting 19:52:05 (5704): No heartbeat from core client for 30 sec - exiting 19:52:06 (5704): No heartbeat from core client for 30 sec - exiting 19:52:07 (5704): No heartbeat from core client for 30 sec - exiting 19:52:08 (5704): No heartbeat from core client for 30 sec - exiting 19:52:09 (5704): No heartbeat from core client for 30 sec - exiting 19:52:10 (5704): No heartbeat from core client for 30 sec - exiting 19:52:11 (5704): No heartbeat from core client for 30 sec - exiting 19:52:12 (5704): No heartbeat from core client for 30 sec - exiting 19:52:13 (5704): No heartbeat from core client for 30 sec - exiting 19:52:14 (5704): No heartbeat from core client for 30 sec - exiting 19:52:15 (5704): No heartbeat from core client for 30 sec - exiting 19:52:16 (5704): No heartbeat from core client for 30 sec - exiting 19:52:17 (5704): No heartbeat from core client for 30 sec - exiting 19:52:18 (5704): No heartbeat from core client for 30 sec - exiting 19:52:19 (5704): No heartbeat from core client for 30 sec - exiting 19:52:20 (5704): No heartbeat from core client for 30 sec - exiting 19:52:21 (5704): No heartbeat from core client for 30 sec - exiting 19:52:22 (5704): No heartbeat from core client for 30 sec - exiting 19:52:23 (5704): No heartbeat from core client for 30 sec - exiting 19:52:24 (5704): No heartbeat from core client for 30 sec - exiting 19:52:25 (5704): No heartbeat from core client for 30 sec - exiting 19:52:26 (5704): No heartbeat from core client for 30 sec - exiting 19:52:27 (5704): No heartbeat from core client for 30 sec - exiting 19:52:28 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:19:52 (4708): No heartbeat from core client for 30 sec - exiting 22:19:54 (4708): No heartbeat from core client for 30 sec - exiting 22:19:55 (4708): No heartbeat from core client for 30 sec - exiting 22:19:56 (4708): No heartbeat from core client for 30 sec - exiting 22:19:57 (4708): No heartbeat from core client for 30 sec - exiting 22:19:58 (4708): No heartbeat from core client for 30 sec - exiting 22:19:59 (4708): No heartbeat from core client for 30 sec - exiting 22:20:00 (4708): No heartbeat from core client for 30 sec - exiting 22:20:01 (4708): No heartbeat from core client for 30 sec - exiting 22:20:02 (4708): No heartbeat from core client for 30 sec - exiting 22:20:03 (4708): No heartbeat from core client for 30 sec - exiting 22:20:04 (4708): No heartbeat from core client for 30 sec - exiting 22:20:05 (4708): No heartbeat from core client for 30 sec - exiting 22:20:06 (4708): No heartbeat from core client for 30 sec - exiting 22:20:07 (4708): No heartbeat from core client for 30 sec - exiting 22:20:08 (4708): No heartbeat from core client for 30 sec - exiting 22:20:09 (4708): No heartbeat from core client for 30 sec - exiting 22:20:10 (4708): No heartbeat from core client for 30 sec - exiting 22:20:11 (4708): No heartbeat from core client for 30 sec - exiting 22:20:12 (4708): No heartbeat from core client for 30 sec - exiting 22:20:13 (4708): No heartbeat from core client for 30 sec - exiting 22:20:14 (4708): No heartbeat from core client for 30 sec - exiting 22:20:15 (4708): No heartbeat from core client for 30 sec - exiting 22:20:16 (4708): No heartbeat from core client for 30 sec - exiting 22:20:17 (4708): No heartbeat from core client for 30 sec - exiting 22:20:18 (4708): No heartbeat from core client for 30 sec - exiting 22:20:19 (4708): No heartbeat from core client for 30 sec - exiting 22:20:20 (4708): No heartbeat from core client for 30 sec - exiting 22:20:21 (4708): No heartbeat from core client for 30 sec - exiting 22:20:22 (4708): No heartbeat from core client for 30 sec - exiting 22:20:23 (4708): No heartbeat from core client for 30 sec - exiting 22:20:24 (4708): No heartbeat from core client for 30 sec - exiting 22:20:25 (4708): No heartbeat from core client for 30 sec - exiting 22:20:26 (4708): No heartbeat from core client for 30 sec - exiting 22:20:27 (4708): No heartbeat from core client for 30 sec - exiting 22:20:28 (4708): No heartbeat from core client for 30 sec - exiting 22:20:29 (4708): No heartbeat from core client for 30 sec - exiting 22:20:30 (4708): No heartbeat from core client for 30 sec - exiting 22:20:31 (4708): No heartbeat from core client for 30 sec - exiting 22:20:32 (4708): No heartbeat from core client for 30 sec - exiting 22:20:33 (4708): No heartbeat from core client for 30 sec - exiting 22:20:34 (4708): No heartbeat from core client for 30 sec - exiting 22:20:35 (4708): No heartbeat from core client for 30 sec - exiting 22:20:36 (4708): No heartbeat from core client for 30 sec - exiting 22:20:37 (4708): No heartbeat from core client for 30 sec - exiting 22:20:38 (4708): No heartbeat from core client for 30 sec - exiting 22:20:39 (4708): No heartbeat from core client for 30 sec - exiting 22:20:40 (4708): No heartbeat from core client for 30 sec - exiting 22:20:41 (4708): No heartbeat from core client for 30 sec - exiting 22:20:42 (4708): No heartbeat from core client for 30 sec - exiting 22:20:43 (4708): No heartbeat from core client for 30 sec - exiting 22:20:44 (4708): No heartbeat from core client for 30 sec - exiting 22:20:45 (4708): No heartbeat from core client for 30 sec - exiting 22:20:46 (4708): No heartbeat from core client for 30 sec - exiting 22:20:47 (4708): No heartbeat from core client for 30 sec - exiting 22:20:48 (4708): No heartbeat from core client for 30 sec - exiting 22:20:49 (4708): No heartbeat from core client for 30 sec - exiting 22:20:50 (4708): No heartbeat from core client for 30 sec - exiting 22:20:51 (4708): No heartbeat from core client for 30 sec - exiting 22:20:52 (4708): No heartbeat from core client for 30 sec - exiting 22:20:53 (4708): No heartbeat from core client for 30 sec - exiting 22:20:54 (4708): No heartbeat from core client for 30 sec - exiting 22:20:55 (4708): No heartbeat from core client for 30 sec - exiting 22:20:56 (4708): No heartbeat from core client for 30 sec - exiting 22:20:57 (4708): No heartbeat from core client for 30 sec - exiting 22:20:58 (4708): No heartbeat from core client for 30 sec - exiting 22:20:59 (4708): No heartbeat from core client for 30 sec - exiting 22:21:00 (4708): No heartbeat from core client for 30 sec - exiting 22:21:01 (4708): No heartbeat from core client for 30 sec - exiting 22:21:02 (4708): No heartbeat from core client for 30 sec - exiting 22:21:03 (4708): No heartbeat from core client for 30 sec - exiting 22:21:04 (4708): No heartbeat from core client for 30 sec - exiting 22:21:05 (4708): No heartbeat from core client for 30 sec - exiting 22:21:06 (4708): No heartbeat from core client for 30 sec - exiting 22:21:07 (4708): No heartbeat from core client for 30 sec - exiting 22:21:08 (4708): No heartbeat from core client for 30 sec - exiting 22:21:09 (4708): No heartbeat from core client for 30 sec - exiting 22:21:10 (4708): No heartbeat from core client for 30 sec - exiting 22:21:11 (4708): No heartbeat from core client for 30 sec - exiting 22:21:12 (4708): No heartbeat from core client for 30 sec - exiting 22:21:13 (4708): No heartbeat from core client for 30 sec - exiting 22:21:14 (4708): No heartbeat from core client for 30 sec - exiting 22:21:15 (4708): No heartbeat from core client for 30 sec - exiting 22:21:16 (4708): No heartbeat from core client for 30 sec - exiting 22:21:17 (4708): No heartbeat from core client for 30 sec - exiting 22:21:18 (4708): No heartbeat from core client for 30 sec - exiting 22:21:19 (4708): No heartbeat from core client for 30 sec - exiting 22:21:20 (4708): No heartbeat from core client for 30 sec - exiting 22:21:21 (4708): No heartbeat from core client for 30 sec - exiting 22:21:22 (4708): No heartbeat from core client for 30 sec - exiting 22:21:23 (4708): No heartbeat from core client for 30 sec - exiting 22:21:24 (4708): No heartbeat from core client for 30 sec - exiting 22:21:25 (4708): No heartbeat from core client for 30 sec - exiting 22:21:26 (4708): No heartbeat from core client for 30 sec - exiting 22:21:27 (4708): No heartbeat from core client for 30 sec - exiting 22:21:28 (4708): No heartbeat from core client for 30 sec - exiting 22:21:29 (4708): No heartbeat from core client for 30 sec - exiting 22:21:30 (4708): No heartbeat from core client for 30 sec - exiting 22:21:31 (4708): No heartbeat from core client for 30 sec - exiting 22:21:32 (4708): No heartbeat from core client for 30 sec - exiting 22:21:33 (4708): No heartbeat from core client for 30 sec - exiting 22:21:34 (4708): No heartbeat from core client for 30 sec - exiting 22:21:35 (4708): No heartbeat from core client for 30 sec - exiting 22:21:36 (4708): No heartbeat from core client for 30 sec - exiting 22:21:37 (4708): No heartbeat from core client for 30 sec - exiting 22:21:38 (4708): No heartbeat from core client for 30 sec - exiting 22:21:39 (4708): No heartbeat from core client for 30 sec - exiting 22:21:40 (4708): No heartbeat from core client for 30 sec - exiting 22:21:41 (4708): No heartbeat from core client for 30 sec - exiting 22:21:42 (4708): No heartbeat from core client for 30 sec - exiting 22:21:43 (4708): No heartbeat from core client for 30 sec - exiting 22:21:44 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:13:42 (5704): No heartbeat from core client for 30 sec - exiting 04:13:43 (5704): No heartbeat from core client for 30 sec - exiting 04:13:44 (5704): No heartbeat from core client for 30 sec - exiting 04:13:45 (5704): No heartbeat from core client for 30 sec - exiting 04:13:46 (5704): No heartbeat from core client for 30 sec - exiting 04:13:47 (5704): No heartbeat from core client for 30 sec - exiting 04:13:48 (5704): No heartbeat from core client for 30 sec - exiting 04:13:49 (5704): No heartbeat from core client for 30 sec - exiting 04:13:50 (5704): No heartbeat from core client for 30 sec - exiting 04:13:51 (5704): No heartbeat from core client for 30 sec - exiting 04:13:52 (5704): No heartbeat from core client for 30 sec - exiting 04:13:53 (5704): No heartbeat from core client for 30 sec - exiting 04:13:54 (5704): No heartbeat from core client for 30 sec - exiting 04:13:55 (5704): No heartbeat from core client for 30 sec - exiting 04:13:56 (5704): No heartbeat from core client for 30 sec - exiting 04:13:57 (5704): No heartbeat from core client for 30 sec - exiting 04:13:58 (5704): No heartbeat from core client for 30 sec - exiting 04:13:59 (5704): No heartbeat from core client for 30 sec - exiting 04:14:00 (5704): No heartbeat from core client for 30 sec - exiting 04:14:01 (5704): No heartbeat from core client for 30 sec - exiting 04:14:02 (5704): No heartbeat from core client for 30 sec - exiting 04:14:03 (5704): No heartbeat from core client for 30 sec - exiting 04:14:04 (5704): No heartbeat from core client for 30 sec - exiting 04:14:05 (5704): No heartbeat from core client for 30 sec - exiting 04:14:06 (5704): No heartbeat from core client for 30 sec - exiting 04:14:07 (5704): No heartbeat from core client for 30 sec - exiting 04:14:08 (5704): No heartbeat from core client for 30 sec - exiting 04:14:09 (5704): No heartbeat from core client for 30 sec - exiting 04:14:10 (5704): No heartbeat from core client for 30 sec - exiting 04:14:11 (5704): No heartbeat from core client for 30 sec - exiting 04:14:12 (5704): No heartbeat from core client for 30 sec - exiting 04:14:13 (5704): No heartbeat from core client for 30 sec - exiting 04:14:14 (5704): No heartbeat from core client for 30 sec - exiting 04:14:15 (5704): No heartbeat from core client for 30 sec - exiting 04:14:16 (5704): No heartbeat from core client for 30 sec - exiting 04:14:17 (5704): No heartbeat from core client for 30 sec - exiting 04:14:18 (5704): No heartbeat from core client for 30 sec - exiting 04:14:19 (5704): No heartbeat from core client for 30 sec - exiting 04:14:20 (5704): No heartbeat from core client for 30 sec - exiting 04:14:21 (5704): No heartbeat from core client for 30 sec - exiting 04:14:22 (5704): No heartbeat from core client for 30 sec - exiting 04:14:23 (5704): No heartbeat from core client for 30 sec - exiting 04:14:24 (5704): No heartbeat from core client for 30 sec - exiting 04:14:25 (5704): No heartbeat from core client for 30 sec - exiting 04:14:26 (5704): No heartbeat from core client for 30 sec - exiting 04:14:27 (5704): No heartbeat from core client for 30 sec - exiting 04:14:28 (5704): No heartbeat from core client for 30 sec - exiting 04:14:29 (5704): No heartbeat from core client for 30 sec - exiting 04:14:30 (5704): No heartbeat from core client for 30 sec - exiting 04:14:31 (5704): No heartbeat from core client for 30 sec - exiting 04:14:32 (5704): No heartbeat from core client for 30 sec - exiting 04:14:33 (5704): No heartbeat from core client for 30 sec - exiting 04:14:34 (5704): No heartbeat from core client for 30 sec - exiting 04:14:35 (5704): No heartbeat from core client for 30 sec - exiting 04:14:36 (5704): No heartbeat from core client for 30 sec - exiting 04:14:37 (5704): No heartbeat from core client for 30 sec - exiting 04:14:38 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:39 (5704): No heartbeat from core client for 30 sec - exiting 04:19:18 (2584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9632, selfPID=9632, iMonCtr=2 18:59:47 (2828): No heartbeat from core client for 30 sec - exiting 18:59:48 (2828): No heartbeat from core client for 30 sec - exiting 18:59:49 (2828): No heartbeat from core client for 30 sec - exiting 18:59:50 (2828): No heartbeat from core client for 30 sec - exiting 18:59:51 (2828): No heartbeat from core client for 30 sec - exiting 18:59:52 (2828): No heartbeat from core client for 30 sec - exiting 18:59:53 (2828): No heartbeat from core client for 30 sec - exiting 18:59:54 (2828): No heartbeat from core client for 30 sec - exiting 18:59:55 (2828): No heartbeat from core client for 30 sec - exiting 18:59:56 (2828): No heartbeat from core client for 30 sec - exiting 18:59:57 (2828): No heartbeat from core client for 30 sec - exiting 18:59:58 (2828): No heartbeat from core client for 30 sec - exiting 18:59:59 (2828): No heartbeat from core client for 30 sec - exiting 19:00:00 (2828): No heartbeat from core client for 30 sec - exiting 19:00:01 (2828): No heartbeat from core client for 30 sec - exiting 19:00:02 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:11 (7776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8132, selfPID=8132, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:53:13 (4868): No heartbeat from core client for 30 sec - exiting 01:53:14 (4868): No heartbeat from core client for 30 sec - exiting 01:53:15 (4868): No heartbeat from core client for 30 sec - exiting 01:53:16 (4868): No heartbeat from core client for 30 sec - exiting 01:53:17 (4868): No heartbeat from core client for 30 sec - exiting 01:53:18 (4868): No heartbeat from core client for 30 sec - exiting 01:53:19 (4868): No heartbeat from core client for 30 sec - exiting 01:53:20 (4868): No heartbeat from core client for 30 sec - exiting 01:53:21 (4868): No heartbeat from core client for 30 sec - exiting 01:53:22 (4868): No heartbeat from core client for 30 sec - exiting 01:53:23 (4868): No heartbeat from core client for 30 sec - exiting 01:53:24 (4868): No heartbeat from core client for 30 sec - exiting 01:53:25 (4868): No heartbeat from core client for 30 sec - exiting 01:53:26 (4868): No heartbeat from core client for 30 sec - exiting 01:53:27 (4868): No heartbeat from core client for 30 sec - exiting 01:53:28 (4868): No heartbeat from core client for 30 sec - exiting 01:53:29 (4868): No heartbeat from core client for 30 sec - exiting 01:53:30 (4868): No heartbeat from core client for 30 sec - exiting 01:53:31 (4868): No heartbeat from core client for 30 sec - exiting 01:53:32 (4868): No heartbeat from core client for 30 sec - exiting 01:53:33 (4868): No heartbeat from core client for 30 sec - exiting 01:53:34 (4868): No heartbeat from core client for 30 sec - exiting 01:53:35 (4868): No heartbeat from core client for 30 sec - exiting 01:53:36 (4868): No heartbeat from core client for 30 sec - exiting 01:53:37 (4868): No heartbeat from core client for 30 sec - exiting 01:53:38 (4868): No heartbeat from core client for 30 sec - exiting 01:53:39 (4868): No heartbeat from core client for 30 sec - exiting 01:53:40 (4868): No heartbeat from core client for 30 sec - exiting 01:53:41 (4868): No heartbeat from core client for 30 sec - exiting 01:53:42 (4868): No heartbeat from core client for 30 sec - exiting 01:53:43 (4868): No heartbeat from core client for 30 sec - exiting 01:53:44 (4868): No heartbeat from core client for 30 sec - exiting 01:53:45 (4868): No heartbeat from core client for 30 sec - exiting 01:53:46 (4868): No heartbeat from core client for 30 sec - exiting 01:53:47 (4868): No heartbeat from core client for 30 sec - exiting 01:53:48 (4868): No heartbeat from core client for 30 sec - exiting 01:53:49 (4868): No heartbeat from core client for 30 sec - exiting 01:53:50 (4868): No heartbeat from core client for 30 sec - exiting 01:53:51 (4868): No heartbeat from core client for 30 sec - exiting 01:53:52 (4868): No heartbeat from core client for 30 sec - exiting 01:53:53 (4868): No heartbeat from core client for 30 sec - exiting 01:53:54 (4868): No heartbeat from core client for 30 sec - exiting 01:53:55 (4868): No heartbeat from core client for 30 sec - exiting 01:53:56 (4868): No heartbeat from core client for 30 sec - exiting 01:53:57 (4868): No heartbeat from core client for 30 sec - exiting 01:53:58 (4868): No heartbeat from core client for 30 sec - exiting 01:53:59 (4868): No heartbeat from core client for 30 sec - exiting 01:54:00 (4868): No heartbeat from core client for 30 sec - exiting 01:54:01 (4868): No heartbeat from core client for 30 sec - exiting 01:54:02 (4868): No heartbeat from core client for 30 sec - exiting 01:54:03 (4868): No heartbeat from core client for 30 sec - exiting 01:54:04 (4868): No heartbeat from core client for 30 sec - exiting 01:54:05 (4868): No heartbeat from core client for 30 sec - exiting 01:54:06 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:42 (3756): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6296, selfPID=6296, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7656, selfPID=1008, iMonCtr=1 Model crash detected, will try to restart... 09:55:09 (5448): No heartbeat from core client for 30 sec - exiting 09:55:10 (5448): No heartbeat from core client for 30 sec - exiting 09:55:11 (5448): No heartbeat from core client for 30 sec - exiting 09:55:12 (5448): No heartbeat from core client for 30 sec - exiting 09:55:13 (5448): No heartbeat from core client for 30 sec - exiting 09:55:14 (5448): No heartbeat from core client for 30 sec - exiting 09:55:15 (5448): No heartbeat from core client for 30 sec - exiting 09:55:16 (5448): No heartbeat from core client for 30 sec - exiting 09:55:17 (5448): No heartbeat from core client for 30 sec - exiting 09:55:18 (5448): No heartbeat from core client for 30 sec - exiting 09:55:19 (5448): No heartbeat from core client for 30 sec - exiting 09:55:20 (5448): No heartbeat from core client for 30 sec - exiting 09:55:21 (5448): No heartbeat from core client for 30 sec - exiting 09:55:22 (5448): No heartbeat from core client for 30 sec - exiting 09:55:23 (5448): No heartbeat from core client for 30 sec - exiting 09:55:24 (5448): No heartbeat from core client for 30 sec - exiting 09:55:25 (5448): No heartbeat from core client for 30 sec - exiting 09:55:26 (5448): No heartbeat from core client for 30 sec - exiting 09:55:27 (5448): No heartbeat from core client for 30 sec - exiting 09:55:28 (5448): No heartbeat from core client for 30 sec - exiting 09:55:29 (5448): No heartbeat from core client for 30 sec - exiting 09:55:30 (5448): No heartbeat from core client for 30 sec - exiting 09:55:31 (5448): No heartbeat from core client for 30 sec - exiting 09:55:32 (5448): No heartbeat from core client for 30 sec - exiting 09:55:33 (5448): No heartbeat from core client for 30 sec - exiting 09:55:34 (5448): No heartbeat from core client for 30 sec - exiting 09:55:35 (5448): No heartbeat from core client for 30 sec - exiting 09:55:36 (5448): No heartbeat from core client for 30 sec - exiting 09:55:37 (5448): No heartbeat from core client for 30 sec - exiting 09:55:38 (5448): No heartbeat from core client for 30 sec - exiting 09:55:39 (5448): No heartbeat from core client for 30 sec - exiting 09:55:40 (5448): No heartbeat from core client for 30 sec - exiting 09:55:41 (5448): No heartbeat from core client for 30 sec - exiting 09:55:42 (5448): No heartbeat from core client for 30 sec - exiting 09:55:43 (5448): No heartbeat from core client for 30 sec - exiting 09:55:44 (5448): No heartbeat from core client for 30 sec - exiting 09:55:45 (5448): No heartbeat from core client for 30 sec - exiting 09:55:46 (5448): No heartbeat from core client for 30 sec - exiting 09:55:47 (5448): No heartbeat from core client for 30 sec - exiting 09:55:48 (5448): No heartbeat from core client for 30 sec - exiting 09:55:49 (5448): No heartbeat from core client for 30 sec - exiting 09:55:50 (5448): No heartbeat from core client for 30 sec - exiting 09:55:51 (5448): No heartbeat from core client for 30 sec - exiting 09:55:52 (5448): No heartbeat from core client for 30 sec - exiting 09:55:53 (5448): No heartbeat from core client for 30 sec - exiting 09:55:54 (5448): No heartbeat from core client for 30 sec - exiting 09:55:55 (5448): No heartbeat from core client for 30 sec - exiting 09:55:56 (5448): No heartbeat from core client for 30 sec - exiting 09:55:57 (5448): No heartbeat from core client for 30 sec - exiting 09:55:58 (5448): No heartbeat from core client for 30 sec - exiting 09:55:59 (5448): No heartbeat from core client for 30 sec - exiting 09:56:00 (5448): No heartbeat from core client for 30 sec - exiting 09:56:01 (5448): No heartbeat from core client for 30 sec - exiting 09:56:02 (5448): No heartbeat from core client for 30 sec - exiting 09:56:03 (5448): No heartbeat from core client for 30 sec - exiting 09:56:04 (5448): No heartbeat from core client for 30 sec - exiting 09:56:05 (5448): No heartbeat from core client for 30 sec - exiting 09:56:06 (5448): No heartbeat from core client for 30 sec - exiting 09:56:07 (5448): No heartbeat from core client for 30 sec - exiting 09:56:08 (5448): No heartbeat from core client for 30 sec - exiting 09:56:09 (5448): No heartbeat from core client for 30 sec - exiting 09:56:10 (5448): No heartbeat from core client for 30 sec - exiting 09:56:11 (5448): No heartbeat from core client for 30 sec - exiting 09:56:12 (5448): No heartbeat from core client for 30 sec - exiting 09:56:13 (5448): No heartbeat from core client for 30 sec - exiting 09:56:14 (5448): No heartbeat from core client for 30 sec - exiting 09:56:15 (5448): No heartbeat from core client for 30 sec - exiting 09:56:16 (5448): No heartbeat from core client for 30 sec - exiting 09:56:17 (5448): No heartbeat from core client for 30 sec - exiting 09:56:18 (5448): No heartbeat from core client for 30 sec - exiting 09:56:19 (5448): No heartbeat from core client for 30 sec - exiting 09:56:20 (5448): No heartbeat from core client for 30 sec - exiting 09:56:21 (5448): No heartbeat from core client for 30 sec - exiting 09:56:22 (5448): No heartbeat from core client for 30 sec - exiting 09:56:23 (5448): No heartbeat from core client for 30 sec - exiting 09:56:24 (5448): No heartbeat from core client for 30 sec - exiting 09:56:25 (5448): No heartbeat from core client for 30 sec - exiting 09:56:26 (5448): No heartbeat from core client for 30 sec - exiting 09:56:27 (5448): No heartbeat from core client for 30 sec - exiting 09:56:28 (5448): No heartbeat from core client for 30 sec - exiting 09:56:29 (5448): No heartbeat from core client for 30 sec - exiting 09:56:30 (5448): No heartbeat from core client for 30 sec - exiting 09:56:31 (5448): No heartbeat from core client for 30 sec - exiting 09:56:32 (5448): No heartbeat from core client for 30 sec - exiting 09:56:33 (5448): No heartbeat from core client for 30 sec - exiting 09:56:34 (5448): No heartbeat from core client for 30 sec - exiting 09:56:35 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 21:30:03 (6200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:51:46 (8232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:02:01 (9496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:05:31 (1596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:24 (7992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:21:37 (9320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8216, selfPID=8216, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8792, selfPID=8792, iMonCtr=2 22:31:10 (8996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4760, selfPID=4760, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 09:56:42 (4128): No heartbeat from core client for 30 sec - exiting 09:56:43 (4128): No heartbeat from core client for 30 sec - exiting 09:56:44 (4128): No heartbeat from core client for 30 sec - exiting 09:56:45 (4128): No heartbeat from core client for 30 sec - exiting 09:56:46 (4128): No heartbeat from core client for 30 sec - exiting 09:56:47 (4128): No heartbeat from core client for 30 sec - exiting 09:56:48 (4128): No heartbeat from core client for 30 sec - exiting 09:56:49 (4128): No heartbeat from core client for 30 sec - exiting 09:56:50 (4128): No heartbeat from core client for 30 sec - exiting 09:56:51 (4128): No heartbeat from core client for 30 sec - exiting 09:56:52 (4128): No heartbeat from core client for 30 sec - exiting 09:56:53 (4128): No heartbeat from core client for 30 sec - exiting 09:56:54 (4128): No heartbeat from core client for 30 sec - exiting 09:56:55 (4128): No heartbeat from core client for 30 sec - exiting 09:56:56 (4128): No heartbeat from core client for 30 sec - exiting 09:56:57 (4128): No heartbeat from core client for 30 sec - exiting 09:56:58 (4128): No heartbeat from core client for 30 sec - exiting 09:56:59 (4128): No heartbeat from core client for 30 sec - exiting 09:57:00 (4128): No heartbeat from core client for 30 sec - exiting 09:57:01 (4128): No heartbeat from core client for 30 sec - exiting 09:57:02 (4128): No heartbeat from core client for 30 sec - exiting 09:57:03 (4128): No heartbeat from core client for 30 sec - exiting 09:57:04 (4128): No heartbeat from core client for 30 sec - exiting 09:57:05 (4128): No heartbeat from core client for 30 sec - exiting 09:57:06 (4128): No heartbeat from core client for 30 sec - exiting 09:57:07 (4128): No heartbeat from core client for 30 sec - exiting 09:57:08 (4128): No heartbeat from core client for 30 sec - exiting 09:57:09 (4128): No heartbeat from core client for 30 sec - exiting 09:57:10 (4128): No heartbeat from core client for 30 sec - exiting 09:57:11 (4128): No heartbeat from core client for 30 sec - exiting 09:57:12 (4128): No heartbeat from core client for 30 sec - exiting 09:57:13 (4128): No heartbeat from core client for 30 sec - exiting 09:57:14 (4128): No heartbeat from core client for 30 sec - exiting 09:57:15 (4128): No heartbeat from core client for 30 sec - exiting 09:57:16 (4128): No heartbeat from core client for 30 sec - exiting 09:57:17 (4128): No heartbeat from core client for 30 sec - exiting 09:57:18 (4128): No heartbeat from core client for 30 sec - exiting 09:57:19 (4128): No heartbeat from core client for 30 sec - exiting 09:57:20 (4128): No heartbeat from core client for 30 sec - exiting 09:57:21 (4128): No heartbeat from core client for 30 sec - exiting 09:57:22 (4128): No heartbeat from core client for 30 sec - exiting 09:57:23 (4128): No heartbeat from core client for 30 sec - exiting 09:57:24 (4128): No heartbeat from core client for 30 sec - exiting 09:57:25 (4128): No heartbeat from core client for 30 sec - exiting 09:57:26 (4128): No heartbeat from core client for 30 sec - exiting 09:57:27 (4128): No heartbeat from core client for 30 sec - exiting 09:57:28 (4128): No heartbeat from core client for 30 sec - exiting 09:57:29 (4128): No heartbeat from core client for 30 sec - exiting 09:57:30 (4128): No heartbeat from core client for 30 sec - exiting 09:57:31 (4128): No heartbeat from core client for 30 sec - exiting 09:57:32 (4128): No heartbeat from core client for 30 sec - exiting 09:57:33 (4128): No heartbeat from core client for 30 sec - exiting 09:57:34 (4128): No heartbeat from core client for 30 sec - exiting 09:57:35 (4128): No heartbeat from core client for 30 sec - exiting 09:57:36 (4128): No heartbeat from core client for 30 sec - exiting 09:57:37 (4128): No heartbeat from core client for 30 sec - exiting 09:57:38 (4128): No heartbeat from core client for 30 sec - exiting 09:57:39 (4128): No heartbeat from core client for 30 sec - exiting 09:57:40 (4128): No heartbeat from core client for 30 sec - exiting 09:57:41 (4128): No heartbeat from core client for 30 sec - exiting 09:57:42 (4128): No heartbeat from core client for 30 sec - exiting 09:57:43 (4128): No heartbeat from core client for 30 sec - exiting 09:57:44 (4128): No heartbeat from core client for 30 sec - exiting 09:57:45 (4128): No heartbeat from core client for 30 sec - exiting 09:57:46 (4128): No heartbeat from core client for 30 sec - exiting 09:57:47 (4128): No heartbeat from core client for 30 sec - exiting 09:57:48 (4128): No heartbeat from core client for 30 sec - exiting 09:57:49 (4128): No heartbeat from core client for 30 sec - exiting 09:57:50 (4128): No heartbeat from core client for 30 sec - exiting 09:57:51 (4128): No heartbeat from core client for 30 sec - exiting 09:57:52 (4128): No heartbeat from core client for 30 sec - exiting 09:57:53 (4128): No heartbeat from core client for 30 sec - exiting 09:57:54 (4128): No heartbeat from core client for 30 sec - exiting 09:57:55 (4128): No heartbeat from core client for 30 sec - exiting 09:57:56 (4128): No heartbeat from core client for 30 sec - exiting 09:57:57 (4128): No heartbeat from core client for 30 sec - exiting 09:57:58 (4128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=284, selfPID=284, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7576, selfPID=7576, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:11:58 (5488): No heartbeat from core client for 30 sec - exiting 12:11:59 (5488): No heartbeat from core client for 30 sec - exiting 12:12:00 (5488): No heartbeat from core client for 30 sec - exiting 12:12:01 (5488): No heartbeat from core client for 30 sec - exiting 12:12:02 (5488): No heartbeat from core client for 30 sec - exiting 12:12:03 (5488): No heartbeat from core client for 30 sec - exiting 12:12:04 (5488): No heartbeat from core client for 30 sec - exiting 12:12:05 (5488): No heartbeat from core client for 30 sec - exiting 12:12:06 (5488): No heartbeat from core client for 30 sec - exiting 12:12:07 (5488): No heartbeat from core client for 30 sec - exiting 12:12:08 (5488): No heartbeat from core client for 30 sec - exiting 12:12:09 (5488): No heartbeat from core client for 30 sec - exiting 12:12:10 (5488): No heartbeat from core client for 30 sec - exiting 12:12:11 (5488): No heartbeat from core client for 30 sec - exiting 12:12:12 (5488): No heartbeat from core client for 30 sec - exiting 12:12:13 (5488): No heartbeat from core client for 30 sec - exiting 12:12:14 (5488): No heartbeat from core client for 30 sec - exiting 12:12:15 (5488): No heartbeat from core client for 30 sec - exiting 12:12:16 (5488): No heartbeat from core client for 30 sec - exiting 12:12:17 (5488): No heartbeat from core client for 30 sec - exiting 12:12:18 (5488): No heartbeat from core client for 30 sec - exiting 12:12:19 (5488): No heartbeat from core client for 30 sec - exiting 12:12:20 (5488): No heartbeat from core client for 30 sec - exiting 12:12:21 (5488): No heartbeat from core client for 30 sec - exiting 12:12:22 (5488): No heartbeat from core client for 30 sec - exiting 12:12:23 (5488): No heartbeat from core client for 30 sec - exiting 12:12:24 (5488): No heartbeat from core client for 30 sec - exiting 12:12:25 (5488): No heartbeat from core client for 30 sec - exiting 12:12:26 (5488): No heartbeat from core client for 30 sec - exiting 12:12:27 (5488): No heartbeat from core client for 30 sec - exiting 12:12:28 (5488): No heartbeat from core client for 30 sec - exiting 12:12:29 (5488): No heartbeat from core client for 30 sec - exiting 12:12:30 (5488): No heartbeat from core client for 30 sec - exiting 12:12:31 (5488): No heartbeat from core client for 30 sec - exiting 12:12:32 (5488): No heartbeat from core client for 30 sec - exiting 12:12:33 (5488): No heartbeat from core client for 30 sec - exiting 12:12:34 (5488): No heartbeat from core client for 30 sec - exiting 12:12:35 (5488): No heartbeat from core client for 30 sec - exiting 12:12:36 (5488): No heartbeat from core client for 30 sec - exiting 12:12:37 (5488): No heartbeat from core client for 30 sec - exiting 12:12:38 (5488): No heartbeat from core client for 30 sec - exiting 12:12:39 (5488): No heartbeat from core client for 30 sec - exiting 12:12:40 (5488): No heartbeat from core client for 30 sec - exiting 12:12:41 (5488): No heartbeat from core client for 30 sec - exiting 12:12:42 (5488): No heartbeat from core client for 30 sec - exiting 12:12:43 (5488): No heartbeat from core client for 30 sec - exiting 12:12:44 (5488): No heartbeat from core client for 30 sec - exiting 12:12:45 (5488): No heartbeat from core client for 30 sec - exiting 12:12:46 (5488): No heartbeat from core client for 30 sec - exiting 12:12:47 (5488): No heartbeat from core client for 30 sec - exiting 12:12:48 (5488): No heartbeat from core client for 30 sec - exiting 12:12:49 (5488): No heartbeat from core client for 30 sec - exiting 12:12:50 (5488): No heartbeat from core client for 30 sec - exiting 12:12:51 (5488): No heartbeat from core client for 30 sec - exiting 12:12:52 (5488): No heartbeat from core client for 30 sec - exiting 12:12:53 (5488): No heartbeat from core client for 30 sec - exiting 12:12:54 (5488): No heartbeat from core client for 30 sec - exiting 12:12:55 (5488): No heartbeat from core client for 30 sec - exiting 12:12:56 (5488): No heartbeat from core client for 30 sec - exiting 12:12:57 (5488): No heartbeat from core client for 30 sec - exiting 12:12:58 (5488): No heartbeat from core client for 30 sec - exiting 12:12:59 (5488): No heartbeat from core client for 30 sec - exiting 12:13:00 (5488): No heartbeat from core client for 30 sec - exiting 12:13:01 (5488): No heartbeat from core client for 30 sec - exiting 12:13:02 (5488): No heartbeat from core client for 30 sec - exiting 12:13:03 (5488): No heartbeat from core client for 30 sec - exiting 12:13:04 (5488): No heartbeat from core client for 30 sec - exiting 12:13:05 (5488): No heartbeat from core client for 30 sec - exiting 12:13:06 (5488): No heartbeat from core client for 30 sec - exiting 12:13:07 (5488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1100, selfPID=5644, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 11 12:15:26 (5644): called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_pnw_znl0_1995_1_006999484_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Feb 2011 17:21:30	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	126,816	405,089	3.1943
19 Feb 2011 14:40:25	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	115,296	367,608	3.1884
18 Feb 2011 08:32:40	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	103,776	331,330	3.1927
17 Feb 2011 21:07:47	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	92,256	294,830	3.1958
17 Feb 2011 16:52:58	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	80,736	258,347	3.1999
16 Feb 2011 15:34:25	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	69,216	222,435	3.2136
15 Feb 2011 19:49:10	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	57,696	185,545	3.2159
14 Feb 2011 19:11:34	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	46,176	148,120	3.2077
14 Feb 2011 06:30:15	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	34,656	110,520	3.1891
13 Feb 2011 19:13:23	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	23,136	73,604	3.1814
13 Feb 2011 13:56:27	1075763	12284321	hadam3p_pnw_znl0_1995_1_006999484_0	11,616	36,619	3.1525