Task 14287451

Name	hadcm3n_o4r6_1980_40_007832730_4
Workunit	7987842
Created	19 Mar 2012, 2:05:48 UTC
Sent	19 Mar 2012, 2:13:38 UTC
Report deadline	18 Jun 2012, 9:40:49 UTC
Received	17 Apr 2012, 11:56:50 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1020546
Run time	25 days 1 hours 22 min 53 sec
CPU time	22 days 18 hours 24 min 42 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:42:24 (9932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:42:27 (9932): No heartbeat from core client for 30 sec - exiting 20:42:28 (9932): No heartbeat from core client for 30 sec - exiting 20:42:29 (9932): No heartbeat from core client for 30 sec - exiting 20:42:30 (9932): No heartbeat from core client for 30 sec - exiting 20:42:31 (9932): No heartbeat from core client for 30 sec - exiting 20:42:32 (9932): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9412, iMonCtr=1 Model crash detected, will try to restart... 14:15:42 (7772): No heartbeat from core client for 30 sec - exiting 14:15:43 (7772): No heartbeat from core client for 30 sec - exiting 14:15:44 (7772): No heartbeat from core client for 30 sec - exiting 14:15:45 (7772): No heartbeat from core client for 30 sec - exiting 14:15:46 (7772): No heartbeat from core client for 30 sec - exiting 14:15:47 (7772): No heartbeat from core client for 30 sec - exiting 14:15:48 (7772): No heartbeat from core client for 30 sec - exiting 14:15:49 (7772): No heartbeat from core client for 30 sec - exiting 14:15:50 (7772): No heartbeat from core client for 30 sec - exiting 14:15:51 (7772): No heartbeat from core client for 30 sec - exiting 14:15:52 (7772): No heartbeat from core client for 30 sec - exiting 14:15:53 (7772): No heartbeat from core client for 30 sec - exiting 14:15:54 (7772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:51 (7604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:51:15 (10072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:51:26 (10072): No heartbeat from core client for 30 sec - exiting 18:51:27 (10072): No heartbeat from core client for 30 sec - exiting 18:51:28 (10072): No heartbeat from core client for 30 sec - exiting 18:51:29 (10072): No heartbeat from core client for 30 sec - exiting 18:51:30 (10072): No heartbeat from core client for 30 sec - exiting 18:52:04 (10072): No heartbeat from core client for 30 sec - exiting 18:52:41 (10072): No heartbeat from core client for 30 sec - exiting 18:52:42 (10072): No heartbeat from core client for 30 sec - exiting 18:52:44 (10072): No heartbeat from core client for 30 sec - exiting 18:52:46 (10072): No heartbeat from core client for 30 sec - exiting 18:52:47 (10072): No heartbeat from core client for 30 sec - exiting 18:52:48 (10072): No heartbeat from core client for 30 sec - exiting 18:52:50 (10072): No heartbeat from core client for 30 sec - exiting 18:52:51 (10072): No heartbeat from core client for 30 sec - exiting 18:52:52 (10072): No heartbeat from core client for 30 sec - exiting 18:52:54 (10072): No heartbeat from core client for 30 sec - exiting 18:52:55 (10072): No heartbeat from core client for 30 sec - exiting 18:52:58 (10072): No heartbeat from core client for 30 sec - exiting 18:52:59 (10072): No heartbeat from core client for 30 sec - exiting 18:53:02 (10072): No heartbeat from core client for 30 sec - exiting 18:53:03 (10072): No heartbeat from core client for 30 sec - exiting 19:02:07 (4604): No heartbeat from core client for 30 sec - exiting 19:02:08 (4604): No heartbeat from core client for 30 sec - exiting 19:02:09 (4604): No heartbeat from core client for 30 sec - exiting 19:02:10 (4604): No heartbeat from core client for 30 sec - exiting 19:02:11 (4604): No heartbeat from core client for 30 sec - exiting 19:02:12 (4604): No heartbeat from core client for 30 sec - exiting 19:02:13 (4604): No heartbeat from core client for 30 sec - exiting 19:02:14 (4604): No heartbeat from core client for 30 sec - exiting 19:02:15 (4604): No heartbeat from core client for 30 sec - exiting 19:02:16 (4604): No heartbeat from core client for 30 sec - exiting 19:02:17 (4604): No heartbeat from core client for 30 sec - exiting 19:02:18 (4604): No heartbeat from core client for 30 sec - exiting 19:02:19 (4604): No heartbeat from core client for 30 sec - exiting 19:02:20 (4604): No heartbeat from core client for 30 sec - exiting 19:02:21 (4604): No heartbeat from core client for 30 sec - exiting 19:02:22 (4604): No heartbeat from core client for 30 sec - exiting 19:02:23 (4604): No heartbeat from core client for 30 sec - exiting 19:02:24 (4604): No heartbeat from core client for 30 sec - exiting 19:02:25 (4604): No heartbeat from core client for 30 sec - exiting 19:02:26 (4604): No heartbeat from core client for 30 sec - exiting 19:02:27 (4604): No heartbeat from core client for 30 sec - exiting 19:02:28 (4604): No heartbeat from core client for 30 sec - exiting 19:02:29 (4604): No heartbeat from core client for 30 sec - exiting 19:02:30 (4604): No heartbeat from core client for 30 sec - exiting 19:02:31 (4604): No heartbeat from core client for 30 sec - exiting 19:02:32 (4604): No heartbeat from core client for 30 sec - exiting 19:02:33 (4604): No heartbeat from core client for 30 sec - exiting 19:02:34 (4604): No heartbeat from core client for 30 sec - exiting 19:02:35 (4604): No heartbeat from core client for 30 sec - exiting 19:02:36 (4604): No heartbeat from core client for 30 sec - exiting 19:02:37 (4604): No heartbeat from core client for 30 sec - exiting 19:02:38 (4604): No heartbeat from core client for 30 sec - exiting 19:02:39 (4604): No heartbeat from core client for 30 sec - exiting 19:02:40 (4604): No heartbeat from core client for 30 sec - exiting 19:02:41 (4604): No heartbeat from core client for 30 sec - exiting 19:02:42 (4604): No heartbeat from core client for 30 sec - exiting 19:02:43 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:02:44 (4604): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:09:51 (8280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:09:56 (8280): No heartbeat from core client for 30 sec - exiting 03:09:57 (8280): No heartbeat from core client for 30 sec - exiting 03:09:58 (8280): No heartbeat from core client for 30 sec - exiting 03:09:59 (8280): No heartbeat from core client for 30 sec - exiting 03:10:00 (8280): No heartbeat from core client for 30 sec - exiting 03:10:01 (8280): No heartbeat from core client for 30 sec - exiting 03:10:02 (8280): No heartbeat from core client for 30 sec - exiting 03:10:03 (8280): No heartbeat from core client for 30 sec - exiting 03:10:04 (8280): No heartbeat from core client for 30 sec - exiting 03:10:05 (8280): No heartbeat from core client for 30 sec - exiting 03:10:06 (8280): No heartbeat from core client for 30 sec - exiting 03:10:07 (8280): No heartbeat from core client for 30 sec - exiting 03:10:08 (8280): No heartbeat from core client for 30 sec - exiting 03:10:09 (8280): No heartbeat from core client for 30 sec - exiting 03:10:10 (8280): No heartbeat from core client for 30 sec - exiting 03:10:11 (8280): No heartbeat from core client for 30 sec - exiting 03:10:12 (8280): No heartbeat from core client for 30 sec - exiting 03:10:13 (8280): No heartbeat from core client for 30 sec - exiting 03:10:14 (8280): No heartbeat from core client for 30 sec - exiting 03:10:15 (8280): No heartbeat from core client for 30 sec - exiting 03:34:05 (8372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:14:41 (2920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:14:45 (2920): No heartbeat from core client for 30 sec - exiting 19:14:47 (2920): No heartbeat from core client for 30 sec - exiting 19:14:48 (2920): No heartbeat from core client for 30 sec - exiting 19:14:49 (2920): No heartbeat from core client for 30 sec - exiting 19:14:50 (2920): No heartbeat from core client for 30 sec - exiting 19:14:51 (2920): No heartbeat from core client for 30 sec - exiting 19:14:52 (2920): No heartbeat from core client for 30 sec - exiting 19:14:53 (2920): No heartbeat from core client for 30 sec - exiting 19:14:54 (2920): No heartbeat from core client for 30 sec - exiting 19:14:55 (2920): No heartbeat from core client for 30 sec - exiting 19:14:56 (2920): No heartbeat from core client for 30 sec - exiting 19:14:57 (2920): No heartbeat from core client for 30 sec - exiting 19:14:58 (2920): No heartbeat from core client for 30 sec - exiting 19:14:59 (2920): No heartbeat from core client for 30 sec - exiting 19:19:33 (5724): No heartbeat from core client for 30 sec - exiting 19:19:40 (5724): No heartbeat from core client for 30 sec - exiting 19:19:41 (5724): No heartbeat from core client for 30 sec - exiting 19:19:42 (5724): No heartbeat from core client for 30 sec - exiting 19:19:43 (5724): No heartbeat from core client for 30 sec - exiting 19:19:44 (5724): No heartbeat from core client for 30 sec - exiting 19:19:45 (5724): No heartbeat from core client for 30 sec - exiting 19:19:46 (5724): No heartbeat from core client for 30 sec - exiting 19:19:47 (5724): No heartbeat from core client for 30 sec - exiting 19:19:48 (5724): No heartbeat from core client for 30 sec - exiting 19:19:49 (5724): No heartbeat from core client for 30 sec - exiting 19:19:50 (5724): No heartbeat from core client for 30 sec - exiting 19:19:51 (5724): No heartbeat from core client for 30 sec - exiting 19:19:52 (5724): No heartbeat from core client for 30 sec - exiting 19:19:53 (5724): No heartbeat from core client for 30 sec - exiting 19:19:54 (5724): No heartbeat from core client for 30 sec - exiting 19:19:55 (5724): No heartbeat from core client for 30 sec - exiting 19:19:56 (5724): No heartbeat from core client for 30 sec - exiting 19:19:57 (5724): No heartbeat from core client for 30 sec - exiting 19:19:58 (5724): No heartbeat from core client for 30 sec - exiting 19:19:59 (5724): No heartbeat from core client for 30 sec - exiting 19:20:00 (5724): No heartbeat from core client for 30 sec - exiting 19:20:01 (5724): No heartbeat from core client for 30 sec - exiting 19:20:02 (5724): No heartbeat from core client for 30 sec - exiting 19:20:03 (5724): No heartbeat from core client for 30 sec - exiting 19:20:04 (5724): No heartbeat from core client for 30 sec - exiting 19:20:05 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:06 (5724): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:13:19 (9620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:00 (4920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:04 (4920): No heartbeat from core client for 30 sec - exiting 19:22:05 (4920): No heartbeat from core client for 30 sec - exiting 19:22:06 (4920): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10260, iMonCtr=1 Model crash detected, will try to restart... 17:04:54 (2192): No heartbeat from core client for 30 sec - exiting 17:04:55 (2192): No heartbeat from core client for 30 sec - exiting 17:04:56 (2192): No heartbeat from core client for 30 sec - exiting 17:04:57 (2192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:12:59 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:48:29 (11188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:34 (11036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:47 (11036): No heartbeat from core client for 30 sec - exiting 23:04:54 (11036): No heartbeat from core client for 30 sec - exiting 23:04:55 (11036): No heartbeat from core client for 30 sec - exiting 23:04:56 (11036): No heartbeat from core client for 30 sec - exiting 23:04:57 (11036): No heartbeat from core client for 30 sec - exiting 23:04:58 (11036): No heartbeat from core client for 30 sec - exiting 23:04:59 (11036): No heartbeat from core client for 30 sec - exiting 23:05:00 (11036): No heartbeat from core client for 30 sec - exiting 23:05:01 (11036): No heartbeat from core client for 30 sec - exiting 23:05:02 (11036): No heartbeat from core client for 30 sec - exiting 23:05:03 (11036): No heartbeat from core client for 30 sec - exiting 23:05:04 (11036): No heartbeat from core client for 30 sec - exiting 23:05:05 (11036): No heartbeat from core client for 30 sec - exiting 23:05:06 (11036): No heartbeat from core client for 30 sec - exiting 23:05:07 (11036): No heartbeat from core client for 30 sec - exiting 23:05:08 (11036): No heartbeat from core client for 30 sec - exiting 23:05:09 (11036): No heartbeat from core client for 30 sec - exiting 23:05:10 (11036): No heartbeat from core client for 30 sec - exiting 23:05:14 (11036): No heartbeat from core client for 30 sec - exiting 23:05:16 (11036): No heartbeat from core client for 30 sec - exiting 23:05:17 (11036): No heartbeat from core client for 30 sec - exiting 23:05:18 (11036): No heartbeat from core client for 30 sec - exiting 23:09:44 (8724): No heartbeat from core client for 30 sec - exiting 23:09:45 (8724): No heartbeat from core client for 30 sec - exiting 23:09:46 (8724): No heartbeat from core client for 30 sec - exiting 23:09:47 (8724): No heartbeat from core client for 30 sec - exiting 23:09:48 (8724): No heartbeat from core client for 30 sec - exiting 23:09:49 (8724): No heartbeat from core client for 30 sec - exiting 23:09:50 (8724): No heartbeat from core client for 30 sec - exiting 23:09:51 (8724): No heartbeat from core client for 30 sec - exiting 23:09:52 (8724): No heartbeat from core client for 30 sec - exiting 23:09:53 (8724): No heartbeat from core client for 30 sec - exiting 23:09:54 (8724): No heartbeat from core client for 30 sec - exiting 23:09:55 (8724): No heartbeat from core client for 30 sec - exiting 23:09:56 (8724): No heartbeat from core client for 30 sec - exiting 23:09:57 (8724): No heartbeat from core client for 30 sec - exiting 23:09:58 (8724): No heartbeat from core client for 30 sec - exiting 23:09:59 (8724): No heartbeat from core client for 30 sec - exiting 23:10:00 (8724): No heartbeat from core client for 30 sec - exiting 23:10:01 (8724): No heartbeat from core client for 30 sec - exiting 23:10:02 (8724): No heartbeat from core client for 30 sec - exiting 23:10:03 (8724): No heartbeat from core client for 30 sec - exiting 23:10:04 (8724): No heartbeat from core client for 30 sec - exiting 23:10:05 (8724): No heartbeat from core client for 30 sec - exiting 23:10:06 (8724): No heartbeat from core client for 30 sec - exiting 23:10:07 (8724): No heartbeat from core client for 30 sec - exiting 23:10:08 (8724): No heartbeat from core client for 30 sec - exiting 23:10:09 (8724): No heartbeat from core client for 30 sec - exiting 23:10:10 (8724): No heartbeat from core client for 30 sec - exiting 23:10:11 (8724): No heartbeat from core client for 30 sec - exiting 23:10:12 (8724): No heartbeat from core client for 30 sec - exiting 23:10:13 (8724): No heartbeat from core client for 30 sec - exiting 23:10:14 (8724): No heartbeat from core client for 30 sec - exiting 23:10:15 (8724): No heartbeat from core client for 30 sec - exiting 23:10:16 (8724): No heartbeat from core client for 30 sec - exiting 23:10:17 (8724): No heartbeat from core client for 30 sec - exiting 23:10:18 (8724): No heartbeat from core client for 30 sec - exiting 23:10:19 (8724): No heartbeat from core client for 30 sec - exiting 23:10:20 (8724): No heartbeat from core client for 30 sec - exiting 23:10:21 (8724): No heartbeat from core client for 30 sec - exiting 23:10:22 (8724): No heartbeat from core client for 30 sec - exiting 23:10:23 (8724): No heartbeat from core client for 30 sec - exiting 23:10:24 (8724): No heartbeat from core client for 30 sec - exiting 23:10:25 (8724): No heartbeat from core client for 30 sec - exiting 23:10:26 (8724): No heartbeat from core client for 30 sec - exiting 23:10:27 (8724): No heartbeat from core client for 30 sec - exiting 23:10:28 (8724): No heartbeat from core client for 30 sec - exiting 23:10:29 (8724): No heartbeat from core client for 30 sec - exiting 23:10:30 (8724): No heartbeat from core client for 30 sec - exiting 23:10:31 (8724): No heartbeat from core client for 30 sec - exiting 23:10:32 (8724): No heartbeat from core client for 30 sec - exiting 23:10:33 (8724): No heartbeat from core client for 30 sec - exiting 23:10:34 (8724): No heartbeat from core client for 30 sec - exiting 23:10:35 (8724): No heartbeat from core client for 30 sec - exiting 23:10:36 (8724): No heartbeat from core client for 30 sec - exiting 23:10:37 (8724): No heartbeat from core client for 30 sec - exiting 23:10:38 (8724): No heartbeat from core client for 30 sec - exiting 23:10:39 (8724): No heartbeat from core client for 30 sec - exiting 23:10:40 (8724): No heartbeat from core client for 30 sec - exiting 23:10:41 (8724): No heartbeat from core client for 30 sec - exiting 23:10:42 (8724): No heartbeat from core client for 30 sec - exiting 23:10:43 (8724): No heartbeat from core client for 30 sec - exiting 23:10:44 (8724): No heartbeat from core client for 30 sec - exiting 23:10:45 (8724): No heartbeat from core client for 30 sec - exiting 23:10:46 (8724): No heartbeat from core client for 30 sec - exiting 23:10:47 (8724): No heartbeat from core client for 30 sec - exiting 23:10:48 (8724): No heartbeat from core client for 30 sec - exiting 23:10:49 (8724): No heartbeat from core client for 30 sec - exiting 23:10:50 (8724): No heartbeat from core client for 30 sec - exiting 23:10:51 (8724): No heartbeat from core client for 30 sec - exiting 23:10:52 (8724): No heartbeat from core client for 30 sec - exiting 23:10:53 (8724): No heartbeat from core client for 30 sec - exiting 23:10:54 (8724): No heartbeat from core client for 30 sec - exiting 23:10:55 (8724): No heartbeat from core client for 30 sec - exiting 23:10:56 (8724): No heartbeat from core client for 30 sec - exiting 23:10:57 (8724): No heartbeat from core client for 30 sec - exiting 23:10:58 (8724): No heartbeat from core client for 30 sec - exiting 23:10:59 (8724): No heartbeat from core client for 30 sec - exiting 23:11:00 (8724): No heartbeat from core client for 30 sec - exiting 23:11:01 (8724): No heartbeat from core client for 30 sec - exiting 23:11:02 (8724): No heartbeat from core client for 30 sec - exiting 23:11:03 (8724): No heartbeat from core client for 30 sec - exiting 23:11:04 (8724): No heartbeat from core client for 30 sec - exiting 23:11:05 (8724): No heartbeat from core client for 30 sec - exiting 23:11:06 (8724): No heartbeat from core client for 30 sec - exiting 23:11:07 (8724): No heartbeat from core client for 30 sec - exiting 23:11:08 (8724): No heartbeat from core client for 30 sec - exiting 23:11:09 (8724): No heartbeat from core client for 30 sec - exiting 23:11:10 (8724): No heartbeat from core client for 30 sec - exiting 23:11:11 (8724): No heartbeat from core client for 30 sec - exiting 23:11:12 (8724): No heartbeat from core client for 30 sec - exiting 23:11:13 (8724): No heartbeat from core client for 30 sec - exiting 23:11:14 (8724): No heartbeat from core client for 30 sec - exiting 23:11:15 (8724): No heartbeat from core client for 30 sec - exiting 23:11:16 (8724): No heartbeat from core client for 30 sec - exiting 23:11:17 (8724): No heartbeat from core client for 30 sec - exiting 23:11:18 (8724): No heartbeat from core client for 30 sec - exiting 23:11:19 (8724): No heartbeat from core client for 30 sec - exiting 23:11:20 (8724): No heartbeat from core client for 30 sec - exiting 23:11:21 (8724): No heartbeat from core client for 30 sec - exiting 23:11:22 (8724): No heartbeat from core client for 30 sec - exiting 23:11:23 (8724): No heartbeat from core client for 30 sec - exiting 23:11:24 (8724): No heartbeat from core client for 30 sec - exiting 23:11:25 (8724): No heartbeat from core client for 30 sec - exiting 23:11:26 (8724): No heartbeat from core client for 30 sec - exiting 23:11:27 (8724): No heartbeat from core client for 30 sec - exiting 23:11:28 (8724): No heartbeat from core client for 30 sec - exiting 23:11:29 (8724): No heartbeat from core client for 30 sec - exiting 23:11:30 (8724): No heartbeat from core client for 30 sec - exiting 23:11:31 (8724): No heartbeat from core client for 30 sec - exiting 23:11:32 (8724): No heartbeat from core client for 30 sec - exiting 23:11:33 (8724): No heartbeat from core client for 30 sec - exiting 23:11:34 (8724): No heartbeat from core client for 30 sec - exiting 23:11:35 (8724): No heartbeat from core client for 30 sec - exiting 23:11:36 (8724): No heartbeat from core client for 30 sec - exiting 23:11:37 (8724): No heartbeat from core client for 30 sec - exiting 23:11:38 (8724): No heartbeat from core client for 30 sec - exiting 23:11:39 (8724): No heartbeat from core client for 30 sec - exiting 23:11:40 (8724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:11:41 (8724): No heartbeat from core client for 30 sec - exiting 23:12:58 (12480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11088, iMonCtr=1 Model crash detected, will try to restart... 06:12:49 (4916): No heartbeat from core client for 30 sec - exiting 06:12:50 (4916): No heartbeat from core client for 30 sec - exiting 06:12:51 (4916): No heartbeat from core client for 30 sec - exiting 06:12:52 (4916): No heartbeat from core client for 30 sec - exiting 06:12:53 (4916): No heartbeat from core client for 30 sec - exiting 06:12:54 (4916): No heartbeat from core client for 30 sec - exiting 06:12:55 (4916): No heartbeat from core client for 30 sec - exiting 06:12:56 (4916): No heartbeat from core client for 30 sec - exiting 06:12:57 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:02:10 (5640): No heartbeat from core client for 30 sec - exiting 20:02:11 (5640): No heartbeat from core client for 30 sec - exiting 20:02:12 (5640): No heartbeat from core client for 30 sec - exiting 20:02:13 (5640): No heartbeat from core client for 30 sec - exiting 20:02:14 (5640): No heartbeat from core client for 30 sec - exiting 20:02:15 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:43:24 (12372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:44:34 (12372): No heartbeat from core client for 30 sec - exiting 21:44:46 (12372): No heartbeat from core client for 30 sec - exiting 21:44:47 (12372): No heartbeat from core client for 30 sec - exiting 21:44:54 (12372): No heartbeat from core client for 30 sec - exiting 21:44:55 (12372): No heartbeat from core client for 30 sec - exiting 21:44:56 (12372): No heartbeat from core client for 30 sec - exiting 21:45:01 (12372): No heartbeat from core client for 30 sec - exiting 21:45:02 (12372): No heartbeat from core client for 30 sec - exiting 21:45:14 (12372): No heartbeat from core client for 30 sec - exiting 21:45:16 (12372): No heartbeat from core client for 30 sec - exiting 21:45:17 (12372): No heartbeat from core client for 30 sec - exiting 21:45:18 (12372): No heartbeat from core client for 30 sec - exiting 21:45:19 (12372): No heartbeat from core client for 30 sec - exiting 21:45:20 (12372): No heartbeat from core client for 30 sec - exiting 21:45:28 (12372): No heartbeat from core client for 30 sec - exiting 21:45:29 (12372): No heartbeat from core client for 30 sec - exiting 22:48:48 (1820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:08:02 (6588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:08:04 (6588): No heartbeat from core client for 30 sec - exiting 12:08:05 (6588): No heartbeat from core client for 30 sec - exiting 12:08:06 (6588): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Apr 2012 10:36:48	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	1,036,800	1,966,438	1.8966
16 Apr 2012 20:44:07	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	1,010,880	1,919,553	1.8989
16 Apr 2012 04:59:32	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	984,960	1,871,108	1.8997
15 Apr 2012 13:44:27	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	959,040	1,821,803	1.8996
11 Apr 2012 19:52:08	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	933,120	1,772,996	1.9001
10 Apr 2012 22:45:26	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	907,200	1,724,810	1.9012
10 Apr 2012 06:00:27	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	881,280	1,676,485	1.9023
09 Apr 2012 12:43:51	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	855,360	1,627,947	1.9032
08 Apr 2012 19:06:33	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	829,440	1,579,283	1.9040
08 Apr 2012 03:17:47	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	803,520	1,530,399	1.9046
07 Apr 2012 12:15:49	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	777,600	1,481,425	1.9051
06 Apr 2012 21:19:18	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	751,680	1,432,794	1.9061
06 Apr 2012 06:28:33	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	725,760	1,383,116	1.9057
05 Apr 2012 14:47:48	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	699,840	1,333,558	1.9055
04 Apr 2012 23:41:10	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	673,920	1,284,366	1.9058
04 Apr 2012 08:52:11	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	648,000	1,235,681	1.9069
03 Apr 2012 17:47:19	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	622,080	1,187,845	1.9095
03 Apr 2012 02:58:14	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	596,160	1,139,688	1.9117
02 Apr 2012 11:21:08	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	570,240	1,091,348	1.9138
01 Apr 2012 20:40:58	1020546	14287451	hadcm3n_o4r6_1980_40_007832730_4	544,320	1,042,099	1.9145