Task 14286734

Name	hadcm3n_yax2_1980_40_007833009_1
Workunit	7988121
Created	19 Mar 2012, 1:05:29 UTC
Sent	19 Mar 2012, 1:06:37 UTC
Report deadline	18 Jun 2012, 8:33:48 UTC
Received	16 Apr 2012, 20:41:53 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1020546
Run time	24 days 22 hours 48 min 3 sec
CPU time	22 days 14 hours 10 min 20 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:42:23 (6604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:42:25 (6604): No heartbeat from core client for 30 sec - exiting 20:42:26 (6604): No heartbeat from core client for 30 sec - exiting 20:42:27 (6604): No heartbeat from core client for 30 sec - exiting 20:42:28 (6604): No heartbeat from core client for 30 sec - exiting 20:42:29 (6604): No heartbeat from core client for 30 sec - exiting 20:42:30 (6604): No heartbeat from core client for 30 sec - exiting 20:42:31 (6604): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4568, iMonCtr=1 Model crash detected, will try to restart... 14:15:42 (6828): No heartbeat from core client for 30 sec - exiting 14:15:43 (6828): No heartbeat from core client for 30 sec - exiting 14:15:44 (6828): No heartbeat from core client for 30 sec - exiting 14:15:45 (6828): No heartbeat from core client for 30 sec - exiting 14:15:46 (6828): No heartbeat from core client for 30 sec - exiting 14:15:47 (6828): No heartbeat from core client for 30 sec - exiting 14:15:48 (6828): No heartbeat from core client for 30 sec - exiting 14:15:49 (6828): No heartbeat from core client for 30 sec - exiting 14:15:50 (6828): No heartbeat from core client for 30 sec - exiting 14:15:51 (6828): No heartbeat from core client for 30 sec - exiting 14:15:52 (6828): No heartbeat from core client for 30 sec - exiting 14:15:53 (6828): No heartbeat from core client for 30 sec - exiting 14:15:54 (6828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:52 (7800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:51:15 (7840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:51:26 (7840): No heartbeat from core client for 30 sec - exiting 18:51:28 (7840): No heartbeat from core client for 30 sec - exiting 18:51:29 (7840): No heartbeat from core client for 30 sec - exiting 18:51:30 (7840): No heartbeat from core client for 30 sec - exiting 18:51:31 (7840): No heartbeat from core client for 30 sec - exiting 18:51:32 (7840): No heartbeat from core client for 30 sec - exiting 18:51:33 (7840): No heartbeat from core client for 30 sec - exiting 18:51:34 (7840): No heartbeat from core client for 30 sec - exiting 18:51:35 (7840): No heartbeat from core client for 30 sec - exiting 18:51:36 (7840): No heartbeat from core client for 30 sec - exiting 18:51:59 (7840): No heartbeat from core client for 30 sec - exiting 18:52:36 (7840): No heartbeat from core client for 30 sec - exiting 18:52:37 (7840): No heartbeat from core client for 30 sec - exiting 18:52:38 (7840): No heartbeat from core client for 30 sec - exiting 18:52:40 (7840): No heartbeat from core client for 30 sec - exiting 18:52:41 (7840): No heartbeat from core client for 30 sec - exiting 18:52:42 (7840): No heartbeat from core client for 30 sec - exiting 18:52:44 (7840): No heartbeat from core client for 30 sec - exiting 18:52:45 (7840): No heartbeat from core client for 30 sec - exiting 18:52:46 (7840): No heartbeat from core client for 30 sec - exiting 18:52:48 (7840): No heartbeat from core client for 30 sec - exiting 18:52:49 (7840): No heartbeat from core client for 30 sec - exiting 18:52:50 (7840): No heartbeat from core client for 30 sec - exiting 18:52:53 (7840): No heartbeat from core client for 30 sec - exiting 18:52:54 (7840): No heartbeat from core client for 30 sec - exiting 18:52:56 (7840): No heartbeat from core client for 30 sec - exiting 18:52:57 (7840): No heartbeat from core client for 30 sec - exiting 18:53:00 (7840): No heartbeat from core client for 30 sec - exiting 18:53:01 (7840): No heartbeat from core client for 30 sec - exiting 18:53:03 (7840): No heartbeat from core client for 30 sec - exiting 18:53:04 (7840): No heartbeat from core client for 30 sec - exiting 19:02:07 (8396): No heartbeat from core client for 30 sec - exiting 19:02:08 (8396): No heartbeat from core client for 30 sec - exiting 19:02:09 (8396): No heartbeat from core client for 30 sec - exiting 19:02:10 (8396): No heartbeat from core client for 30 sec - exiting 19:02:11 (8396): No heartbeat from core client for 30 sec - exiting 19:02:12 (8396): No heartbeat from core client for 30 sec - exiting 19:02:13 (8396): No heartbeat from core client for 30 sec - exiting 19:02:14 (8396): No heartbeat from core client for 30 sec - exiting 19:02:15 (8396): No heartbeat from core client for 30 sec - exiting 19:02:16 (8396): No heartbeat from core client for 30 sec - exiting 19:02:17 (8396): No heartbeat from core client for 30 sec - exiting 19:02:18 (8396): No heartbeat from core client for 30 sec - exiting 19:02:19 (8396): No heartbeat from core client for 30 sec - exiting 19:02:20 (8396): No heartbeat from core client for 30 sec - exiting 19:02:21 (8396): No heartbeat from core client for 30 sec - exiting 19:02:22 (8396): No heartbeat from core client for 30 sec - exiting 19:02:23 (8396): No heartbeat from core client for 30 sec - exiting 19:02:24 (8396): No heartbeat from core client for 30 sec - exiting 19:02:25 (8396): No heartbeat from core client for 30 sec - exiting 19:02:26 (8396): No heartbeat from core client for 30 sec - exiting 19:02:27 (8396): No heartbeat from core client for 30 sec - exiting 19:02:28 (8396): No heartbeat from core client for 30 sec - exiting 19:02:29 (8396): No heartbeat from core client for 30 sec - exiting 19:02:30 (8396): No heartbeat from core client for 30 sec - exiting 19:02:31 (8396): No heartbeat from core client for 30 sec - exiting 19:02:32 (8396): No heartbeat from core client for 30 sec - exiting 19:02:33 (8396): No heartbeat from core client for 30 sec - exiting 19:02:34 (8396): No heartbeat from core client for 30 sec - exiting 19:02:35 (8396): No heartbeat from core client for 30 sec - exiting 19:02:36 (8396): No heartbeat from core client for 30 sec - exiting 19:02:37 (8396): No heartbeat from core client for 30 sec - exiting 19:02:38 (8396): No heartbeat from core client for 30 sec - exiting 19:02:39 (8396): No heartbeat from core client for 30 sec - exiting 19:02:40 (8396): No heartbeat from core client for 30 sec - exiting 19:02:41 (8396): No heartbeat from core client for 30 sec - exiting 19:02:42 (8396): No heartbeat from core client for 30 sec - exiting 19:02:43 (8396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:09:51 (8496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:09:56 (8496): No heartbeat from core client for 30 sec - exiting 03:09:57 (8496): No heartbeat from core client for 30 sec - exiting 03:09:58 (8496): No heartbeat from core client for 30 sec - exiting 03:09:59 (8496): No heartbeat from core client for 30 sec - exiting 03:10:00 (8496): No heartbeat from core client for 30 sec - exiting 03:10:01 (8496): No heartbeat from core client for 30 sec - exiting 03:10:02 (8496): No heartbeat from core client for 30 sec - exiting 03:10:03 (8496): No heartbeat from core client for 30 sec - exiting 03:10:04 (8496): No heartbeat from core client for 30 sec - exiting 03:10:05 (8496): No heartbeat from core client for 30 sec - exiting 03:10:06 (8496): No heartbeat from core client for 30 sec - exiting 03:10:07 (8496): No heartbeat from core client for 30 sec - exiting 03:10:08 (8496): No heartbeat from core client for 30 sec - exiting 03:10:09 (8496): No heartbeat from core client for 30 sec - exiting 03:10:10 (8496): No heartbeat from core client for 30 sec - exiting 03:10:11 (8496): No heartbeat from core client for 30 sec - exiting 03:10:12 (8496): No heartbeat from core client for 30 sec - exiting 03:10:13 (8496): No heartbeat from core client for 30 sec - exiting 03:10:14 (8496): No heartbeat from core client for 30 sec - exiting 03:10:15 (8496): No heartbeat from core client for 30 sec - exiting 03:34:04 (9092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:14:41 (460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:14:45 (460): No heartbeat from core client for 30 sec - exiting 19:14:46 (460): No heartbeat from core client for 30 sec - exiting 19:14:47 (460): No heartbeat from core client for 30 sec - exiting 19:14:48 (460): No heartbeat from core client for 30 sec - exiting 19:14:49 (460): No heartbeat from core client for 30 sec - exiting 19:14:50 (460): No heartbeat from core client for 30 sec - exiting 19:14:51 (460): No heartbeat from core client for 30 sec - exiting 19:14:52 (460): No heartbeat from core client for 30 sec - exiting 19:14:53 (460): No heartbeat from core client for 30 sec - exiting 19:14:54 (460): No heartbeat from core client for 30 sec - exiting 19:14:55 (460): No heartbeat from core client for 30 sec - exiting 19:14:56 (460): No heartbeat from core client for 30 sec - exiting 19:14:57 (460): No heartbeat from core client for 30 sec - exiting 19:14:58 (460): No heartbeat from core client for 30 sec - exiting 19:15:00 (460): No heartbeat from core client for 30 sec - exiting 19:15:01 (460): No heartbeat from core client for 30 sec - exiting 19:15:02 (460): No heartbeat from core client for 30 sec - exiting 19:15:03 (460): No heartbeat from core client for 30 sec - exiting 19:15:04 (460): No heartbeat from core client for 30 sec - exiting 19:19:22 (9496): No heartbeat from core client for 30 sec - exiting 19:19:25 (9496): No heartbeat from core client for 30 sec - exiting 19:19:26 (9496): No heartbeat from core client for 30 sec - exiting 19:19:27 (9496): No heartbeat from core client for 30 sec - exiting 19:19:28 (9496): No heartbeat from core client for 30 sec - exiting 19:19:29 (9496): No heartbeat from core client for 30 sec - exiting 19:19:30 (9496): No heartbeat from core client for 30 sec - exiting 19:19:31 (9496): No heartbeat from core client for 30 sec - exiting 19:19:32 (9496): No heartbeat from core client for 30 sec - exiting 19:19:33 (9496): No heartbeat from core client for 30 sec - exiting 19:19:34 (9496): No heartbeat from core client for 30 sec - exiting 19:19:35 (9496): No heartbeat from core client for 30 sec - exiting 19:19:36 (9496): No heartbeat from core client for 30 sec - exiting 19:19:37 (9496): No heartbeat from core client for 30 sec - exiting 19:19:38 (9496): No heartbeat from core client for 30 sec - exiting 19:19:39 (9496): No heartbeat from core client for 30 sec - exiting 19:19:40 (9496): No heartbeat from core client for 30 sec - exiting 19:19:41 (9496): No heartbeat from core client for 30 sec - exiting 19:19:42 (9496): No heartbeat from core client for 30 sec - exiting 19:19:43 (9496): No heartbeat from core client for 30 sec - exiting 19:19:44 (9496): No heartbeat from core client for 30 sec - exiting 19:19:45 (9496): No heartbeat from core client for 30 sec - exiting 19:19:46 (9496): No heartbeat from core client for 30 sec - exiting 19:19:47 (9496): No heartbeat from core client for 30 sec - exiting 19:19:48 (9496): No heartbeat from core client for 30 sec - exiting 19:19:49 (9496): No heartbeat from core client for 30 sec - exiting 19:19:50 (9496): No heartbeat from core client for 30 sec - exiting 19:19:51 (9496): No heartbeat from core client for 30 sec - exiting 19:19:52 (9496): No heartbeat from core client for 30 sec - exiting 19:19:53 (9496): No heartbeat from core client for 30 sec - exiting 19:19:54 (9496): No heartbeat from core client for 30 sec - exiting 19:19:55 (9496): No heartbeat from core client for 30 sec - exiting 19:19:56 (9496): No heartbeat from core client for 30 sec - exiting 19:19:57 (9496): No heartbeat from core client for 30 sec - exiting 19:19:58 (9496): No heartbeat from core client for 30 sec - exiting 19:19:59 (9496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:00 (9496): No heartbeat from core client for 30 sec - exiting 19:20:01 (9496): No heartbeat from core client for 30 sec - exiting 19:20:02 (9496): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:13:18 (10884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:00 (1272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:04 (1272): No heartbeat from core client for 30 sec - exiting 19:22:05 (1272): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11104, iMonCtr=1 Model crash detected, will try to restart... 17:04:54 (3292): No heartbeat from core client for 30 sec - exiting 17:04:55 (3292): No heartbeat from core client for 30 sec - exiting 17:04:56 (3292): No heartbeat from core client for 30 sec - exiting 17:04:57 (3292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:12:59 (5320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:48:30 (11300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:33 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:47 (6356): No heartbeat from core client for 30 sec - exiting 23:04:55 (6356): No heartbeat from core client for 30 sec - exiting 23:04:56 (6356): No heartbeat from core client for 30 sec - exiting 23:04:57 (6356): No heartbeat from core client for 30 sec - exiting 23:04:58 (6356): No heartbeat from core client for 30 sec - exiting 23:04:59 (6356): No heartbeat from core client for 30 sec - exiting 23:05:00 (6356): No heartbeat from core client for 30 sec - exiting 23:05:01 (6356): No heartbeat from core client for 30 sec - exiting 23:05:02 (6356): No heartbeat from core client for 30 sec - exiting 23:05:03 (6356): No heartbeat from core client for 30 sec - exiting 23:05:04 (6356): No heartbeat from core client for 30 sec - exiting 23:05:05 (6356): No heartbeat from core client for 30 sec - exiting 23:05:06 (6356): No heartbeat from core client for 30 sec - exiting 23:05:07 (6356): No heartbeat from core client for 30 sec - exiting 23:05:08 (6356): No heartbeat from core client for 30 sec - exiting 23:05:09 (6356): No heartbeat from core client for 30 sec - exiting 23:05:10 (6356): No heartbeat from core client for 30 sec - exiting 23:05:11 (6356): No heartbeat from core client for 30 sec - exiting 23:05:12 (6356): No heartbeat from core client for 30 sec - exiting 23:05:14 (6356): No heartbeat from core client for 30 sec - exiting 23:05:16 (6356): No heartbeat from core client for 30 sec - exiting 23:05:17 (6356): No heartbeat from core client for 30 sec - exiting 23:05:18 (6356): No heartbeat from core client for 30 sec - exiting 23:09:44 (8712): No heartbeat from core client for 30 sec - exiting 23:09:45 (8712): No heartbeat from core client for 30 sec - exiting 23:09:46 (8712): No heartbeat from core client for 30 sec - exiting 23:09:47 (8712): No heartbeat from core client for 30 sec - exiting 23:09:48 (8712): No heartbeat from core client for 30 sec - exiting 23:09:49 (8712): No heartbeat from core client for 30 sec - exiting 23:09:50 (8712): No heartbeat from core client for 30 sec - exiting 23:09:51 (8712): No heartbeat from core client for 30 sec - exiting 23:09:52 (8712): No heartbeat from core client for 30 sec - exiting 23:09:53 (8712): No heartbeat from core client for 30 sec - exiting 23:09:54 (8712): No heartbeat from core client for 30 sec - exiting 23:09:55 (8712): No heartbeat from core client for 30 sec - exiting 23:09:56 (8712): No heartbeat from core client for 30 sec - exiting 23:09:57 (8712): No heartbeat from core client for 30 sec - exiting 23:09:58 (8712): No heartbeat from core client for 30 sec - exiting 23:09:59 (8712): No heartbeat from core client for 30 sec - exiting 23:10:00 (8712): No heartbeat from core client for 30 sec - exiting 23:10:01 (8712): No heartbeat from core client for 30 sec - exiting 23:10:02 (8712): No heartbeat from core client for 30 sec - exiting 23:10:03 (8712): No heartbeat from core client for 30 sec - exiting 23:10:04 (8712): No heartbeat from core client for 30 sec - exiting 23:10:05 (8712): No heartbeat from core client for 30 sec - exiting 23:10:06 (8712): No heartbeat from core client for 30 sec - exiting 23:10:07 (8712): No heartbeat from core client for 30 sec - exiting 23:10:08 (8712): No heartbeat from core client for 30 sec - exiting 23:10:09 (8712): No heartbeat from core client for 30 sec - exiting 23:10:10 (8712): No heartbeat from core client for 30 sec - exiting 23:10:11 (8712): No heartbeat from core client for 30 sec - exiting 23:10:12 (8712): No heartbeat from core client for 30 sec - exiting 23:10:13 (8712): No heartbeat from core client for 30 sec - exiting 23:10:14 (8712): No heartbeat from core client for 30 sec - exiting 23:10:15 (8712): No heartbeat from core client for 30 sec - exiting 23:10:16 (8712): No heartbeat from core client for 30 sec - exiting 23:10:17 (8712): No heartbeat from core client for 30 sec - exiting 23:10:18 (8712): No heartbeat from core client for 30 sec - exiting 23:10:19 (8712): No heartbeat from core client for 30 sec - exiting 23:10:20 (8712): No heartbeat from core client for 30 sec - exiting 23:10:21 (8712): No heartbeat from core client for 30 sec - exiting 23:10:22 (8712): No heartbeat from core client for 30 sec - exiting 23:10:23 (8712): No heartbeat from core client for 30 sec - exiting 23:10:24 (8712): No heartbeat from core client for 30 sec - exiting 23:10:25 (8712): No heartbeat from core client for 30 sec - exiting 23:10:26 (8712): No heartbeat from core client for 30 sec - exiting 23:10:27 (8712): No heartbeat from core client for 30 sec - exiting 23:10:28 (8712): No heartbeat from core client for 30 sec - exiting 23:10:29 (8712): No heartbeat from core client for 30 sec - exiting 23:10:30 (8712): No heartbeat from core client for 30 sec - exiting 23:10:31 (8712): No heartbeat from core client for 30 sec - exiting 23:10:32 (8712): No heartbeat from core client for 30 sec - exiting 23:10:33 (8712): No heartbeat from core client for 30 sec - exiting 23:10:34 (8712): No heartbeat from core client for 30 sec - exiting 23:10:35 (8712): No heartbeat from core client for 30 sec - exiting 23:10:36 (8712): No heartbeat from core client for 30 sec - exiting 23:10:37 (8712): No heartbeat from core client for 30 sec - exiting 23:10:38 (8712): No heartbeat from core client for 30 sec - exiting 23:10:39 (8712): No heartbeat from core client for 30 sec - exiting 23:10:40 (8712): No heartbeat from core client for 30 sec - exiting 23:10:41 (8712): No heartbeat from core client for 30 sec - exiting 23:10:42 (8712): No heartbeat from core client for 30 sec - exiting 23:10:43 (8712): No heartbeat from core client for 30 sec - exiting 23:10:44 (8712): No heartbeat from core client for 30 sec - exiting 23:10:45 (8712): No heartbeat from core client for 30 sec - exiting 23:10:46 (8712): No heartbeat from core client for 30 sec - exiting 23:10:47 (8712): No heartbeat from core client for 30 sec - exiting 23:10:48 (8712): No heartbeat from core client for 30 sec - exiting 23:10:49 (8712): No heartbeat from core client for 30 sec - exiting 23:10:50 (8712): No heartbeat from core client for 30 sec - exiting 23:10:51 (8712): No heartbeat from core client for 30 sec - exiting 23:10:52 (8712): No heartbeat from core client for 30 sec - exiting 23:10:53 (8712): No heartbeat from core client for 30 sec - exiting 23:10:54 (8712): No heartbeat from core client for 30 sec - exiting 23:10:55 (8712): No heartbeat from core client for 30 sec - exiting 23:10:56 (8712): No heartbeat from core client for 30 sec - exiting 23:10:57 (8712): No heartbeat from core client for 30 sec - exiting 23:10:58 (8712): No heartbeat from core client for 30 sec - exiting 23:10:59 (8712): No heartbeat from core client for 30 sec - exiting 23:11:00 (8712): No heartbeat from core client for 30 sec - exiting 23:11:01 (8712): No heartbeat from core client for 30 sec - exiting 23:11:02 (8712): No heartbeat from core client for 30 sec - exiting 23:11:03 (8712): No heartbeat from core client for 30 sec - exiting 23:11:04 (8712): No heartbeat from core client for 30 sec - exiting 23:11:05 (8712): No heartbeat from core client for 30 sec - exiting 23:11:06 (8712): No heartbeat from core client for 30 sec - exiting 23:11:07 (8712): No heartbeat from core client for 30 sec - exiting 23:11:08 (8712): No heartbeat from core client for 30 sec - exiting 23:11:09 (8712): No heartbeat from core client for 30 sec - exiting 23:11:10 (8712): No heartbeat from core client for 30 sec - exiting 23:11:11 (8712): No heartbeat from core client for 30 sec - exiting 23:11:12 (8712): No heartbeat from core client for 30 sec - exiting 23:11:13 (8712): No heartbeat from core client for 30 sec - exiting 23:11:14 (8712): No heartbeat from core client for 30 sec - exiting 23:11:15 (8712): No heartbeat from core client for 30 sec - exiting 23:11:16 (8712): No heartbeat from core client for 30 sec - exiting 23:11:17 (8712): No heartbeat from core client for 30 sec - exiting 23:11:18 (8712): No heartbeat from core client for 30 sec - exiting 23:11:19 (8712): No heartbeat from core client for 30 sec - exiting 23:11:20 (8712): No heartbeat from core client for 30 sec - exiting 23:11:21 (8712): No heartbeat from core client for 30 sec - exiting 23:11:22 (8712): No heartbeat from core client for 30 sec - exiting 23:11:23 (8712): No heartbeat from core client for 30 sec - exiting 23:11:24 (8712): No heartbeat from core client for 30 sec - exiting 23:11:25 (8712): No heartbeat from core client for 30 sec - exiting 23:11:26 (8712): No heartbeat from core client for 30 sec - exiting 23:11:27 (8712): No heartbeat from core client for 30 sec - exiting 23:11:28 (8712): No heartbeat from core client for 30 sec - exiting 23:11:29 (8712): No heartbeat from core client for 30 sec - exiting 23:11:30 (8712): No heartbeat from core client for 30 sec - exiting 23:11:31 (8712): No heartbeat from core client for 30 sec - exiting 23:11:32 (8712): No heartbeat from core client for 30 sec - exiting 23:11:33 (8712): No heartbeat from core client for 30 sec - exiting 23:11:34 (8712): No heartbeat from core client for 30 sec - exiting 23:11:35 (8712): No heartbeat from core client for 30 sec - exiting 23:11:36 (8712): No heartbeat from core client for 30 sec - exiting 23:11:37 (8712): No heartbeat from core client for 30 sec - exiting 23:11:38 (8712): No heartbeat from core client for 30 sec - exiting 23:11:39 (8712): No heartbeat from core client for 30 sec - exiting 23:11:40 (8712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:58 (13140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2328, iMonCtr=1 Model crash detected, will try to restart... 06:12:49 (5908): No heartbeat from core client for 30 sec - exiting 06:12:50 (5908): No heartbeat from core client for 30 sec - exiting 06:12:51 (5908): No heartbeat from core client for 30 sec - exiting 06:12:52 (5908): No heartbeat from core client for 30 sec - exiting 06:12:53 (5908): No heartbeat from core client for 30 sec - exiting 06:12:54 (5908): No heartbeat from core client for 30 sec - exiting 06:12:55 (5908): No heartbeat from core client for 30 sec - exiting 06:12:56 (5908): No heartbeat from core client for 30 sec - exiting 06:12:57 (5908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Apr 2012 10:30:49	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	1,036,800	1,951,184	1.8819
15 Apr 2012 19:37:31	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	1,010,880	1,902,461	1.8820
15 Apr 2012 05:15:38	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	984,960	1,853,475	1.8818
11 Apr 2012 11:20:57	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	959,040	1,805,797	1.8829
10 Apr 2012 21:43:19	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	933,120	1,757,573	1.8835
10 Apr 2012 07:00:30	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	907,200	1,709,680	1.8846
09 Apr 2012 15:43:27	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	881,280	1,661,415	1.8852
09 Apr 2012 01:11:17	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	855,360	1,613,157	1.8859
08 Apr 2012 10:35:49	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	829,440	1,564,711	1.8865
07 Apr 2012 20:29:53	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	803,520	1,515,990	1.8867
07 Apr 2012 05:08:06	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	777,600	1,467,467	1.8872
06 Apr 2012 14:22:16	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	751,680	1,418,816	1.8875
05 Apr 2012 23:25:59	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	725,760	1,369,204	1.8866
05 Apr 2012 08:25:39	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	699,840	1,320,529	1.8869
04 Apr 2012 17:17:26	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	673,920	1,271,324	1.8865
04 Apr 2012 02:35:40	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	648,000	1,223,541	1.8882
03 Apr 2012 12:07:55	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	622,080	1,175,871	1.8902
02 Apr 2012 21:32:20	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	596,160	1,128,403	1.8928
02 Apr 2012 06:10:46	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	570,240	1,080,008	1.8940
01 Apr 2012 15:26:29	1020546	14286734	hadcm3n_yax2_1980_40_007833009_1	544,320	1,031,428	1.8949