Task 15804975

Name	hadcm3n_o0eh_2020_40_008376112_0
Workunit	8526971
Created	30 May 2013, 1:49:08 UTC
Sent	30 May 2013, 1:53:50 UTC
Report deadline	29 Aug 2013, 9:21:01 UTC
Received	11 Jun 2013, 12:27:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1230144
Run time	11 days 3 hours 6 min 5 sec
CPU time	10 days 8 hours 17 min 14 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.22 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> 22:59:05 (3660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:19 (4976): No heartbeat from core client for 30 sec - exiting 23:04:20 (4976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:29:59 (4936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:30:01 (4936): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 23:32:14 (4936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 00:05:38 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:57:31 (5200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:11:53 (1412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:18:57 (812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:31:43 (5136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:05 (1240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:06 (1240): No heartbeat from core client for 30 sec - exiting 03:50:04 (4280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:50:05 (4280): No heartbeat from core client for 30 sec - exiting 03:50:07 (4280): No heartbeat from core client for 30 sec - exiting 03:50:08 (4280): No heartbeat from core client for 30 sec - exiting 03:50:09 (4280): No heartbeat from core client for 30 sec - exiting 03:50:10 (4280): No heartbeat from core client for 30 sec - exiting 03:50:11 (4280): No heartbeat from core client for 30 sec - exiting 03:50:12 (4280): No heartbeat from core client for 30 sec - exiting 03:50:13 (4280): No heartbeat from core client for 30 sec - exiting 03:50:15 (4280): No heartbeat from core client for 30 sec - exiting 03:53:21 (3544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:53:45 (3544): No heartbeat from core client for 30 sec - exiting 03:53:46 (3544): No heartbeat from core client for 30 sec - exiting 03:53:47 (3544): No heartbeat from core client for 30 sec - exiting 03:53:48 (3544): No heartbeat from core client for 30 sec - exiting 03:53:49 (3544): No heartbeat from core client for 30 sec - exiting 03:53:50 (3544): No heartbeat from core client for 30 sec - exiting 03:53:51 (3544): No heartbeat from core client for 30 sec - exiting 03:53:52 (3544): No heartbeat from core client for 30 sec - exiting 03:53:53 (3544): No heartbeat from core client for 30 sec - exiting 03:53:55 (3544): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 04:03:21 (2568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:45:12 (5072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:02:57 (4704): No heartbeat from core client for 30 sec - exiting 05:02:58 (4704): No heartbeat from core client for 30 sec - exiting 05:03:50 (4704): No heartbeat from core client for 30 sec - exiting 05:03:51 (4704): No heartbeat from core client for 30 sec - exiting 05:03:52 (4704): No heartbeat from core client for 30 sec - exiting 05:03:53 (4704): No heartbeat from core client for 30 sec - exiting 05:03:54 (4704): No heartbeat from core client for 30 sec - exiting 05:03:55 (4704): No heartbeat from core client for 30 sec - exiting 05:03:56 (4704): No heartbeat from core client for 30 sec - exiting 05:03:57 (4704): No heartbeat from core client for 30 sec - exiting 05:03:58 (4704): No heartbeat from core client for 30 sec - exiting 05:03:59 (4704): No heartbeat from core client for 30 sec - exiting 05:04:01 (4704): No heartbeat from core client for 30 sec - exiting 05:04:02 (4704): No heartbeat from core client for 30 sec - exiting 05:04:03 (4704): No heartbeat from core client for 30 sec - exiting 05:04:04 (4704): No heartbeat from core client for 30 sec - exiting 05:04:05 (4704): No heartbeat from core client for 30 sec - exiting 05:04:06 (4704): No heartbeat from core client for 30 sec - exiting 05:04:07 (4704): No heartbeat from core client for 30 sec - exiting 05:04:08 (4704): No heartbeat from core client for 30 sec - exiting 05:04:09 (4704): No heartbeat from core client for 30 sec - exiting 05:04:10 (4704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:07:16 (4140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:10:27 (4380): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:19:34 (3532): No heartbeat from core client for 30 sec - exiting 18:19:35 (3532): No heartbeat from core client for 30 sec - exiting 18:19:37 (3532): No heartbeat from core client for 30 sec - exiting 18:19:38 (3532): No heartbeat from core client for 30 sec - exiting 18:19:39 (3532): No heartbeat from core client for 30 sec - exiting 18:19:40 (3532): No heartbeat from core client for 30 sec - exiting 18:19:41 (3532): No heartbeat from core client for 30 sec - exiting 18:19:42 (3532): No heartbeat from core client for 30 sec - exiting 18:19:43 (3532): No heartbeat from core client for 30 sec - exiting 18:19:44 (3532): No heartbeat from core client for 30 sec - exiting 18:19:45 (3532): No heartbeat from core client for 30 sec - exiting 18:19:47 (3532): No heartbeat from core client for 30 sec - exiting 18:19:48 (3532): No heartbeat from core client for 30 sec - exiting 18:19:49 (3532): No heartbeat from core client for 30 sec - exiting 18:19:50 (3532): No heartbeat from core client for 30 sec - exiting 18:19:51 (3532): No heartbeat from core client for 30 sec - exiting 18:19:52 (3532): No heartbeat from core client for 30 sec - exiting 18:19:53 (3532): No heartbeat from core client for 30 sec - exiting 18:19:54 (3532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:21:05 (5044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:28:59 (1272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:01:01 (4380): No heartbeat from core client for 30 sec - exiting 05:01:02 (4380): No heartbeat from core client for 30 sec - exiting 05:01:04 (4380): No heartbeat from core client for 30 sec - exiting 05:01:05 (4380): No heartbeat from core client for 30 sec - exiting 05:01:06 (4380): No heartbeat from core client for 30 sec - exiting 05:01:07 (4380): No heartbeat from core client for 30 sec - exiting 05:01:08 (4380): No heartbeat from core client for 30 sec - exiting 05:01:09 (4380): No heartbeat from core client for 30 sec - exiting 05:01:10 (4380): No heartbeat from core client for 30 sec - exiting 05:01:11 (4380): No heartbeat from core client for 30 sec - exiting 05:01:12 (4380): No heartbeat from core client for 30 sec - exiting 05:01:13 (4380): No heartbeat from core client for 30 sec - exiting 05:01:14 (4380): No heartbeat from core client for 30 sec - exiting 05:01:16 (4380): No heartbeat from core client for 30 sec - exiting 05:01:17 (4380): No heartbeat from core client for 30 sec - exiting 05:01:18 (4380): No heartbeat from core client for 30 sec - exiting 05:01:19 (4380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 19:49:14 (4364): No heartbeat from core client for 30 sec - exiting 19:49:15 (4364): No heartbeat from core client for 30 sec - exiting 19:49:16 (4364): No heartbeat from core client for 30 sec - exiting 19:49:17 (4364): No heartbeat from core client for 30 sec - exiting 19:49:18 (4364): No heartbeat from core client for 30 sec - exiting 19:49:19 (4364): No heartbeat from core client for 30 sec - exiting 19:49:21 (4364): No heartbeat from core client for 30 sec - exiting 19:49:22 (4364): No heartbeat from core client for 30 sec - exiting 19:49:23 (4364): No heartbeat from core client for 30 sec - exiting 19:49:24 (4364): No heartbeat from core client for 30 sec - exiting 19:49:25 (4364): No heartbeat from core client for 30 sec - exiting 19:49:26 (4364): No heartbeat from core client for 30 sec - exiting 19:49:27 (4364): No heartbeat from core client for 30 sec - exiting 19:49:28 (4364): No heartbeat from core client for 30 sec - exiting 19:49:29 (4364): No heartbeat from core client for 30 sec - exiting 19:49:30 (4364): No heartbeat from core client for 30 sec - exiting 19:49:32 (4364): No heartbeat from core client for 30 sec - exiting 19:49:33 (4364): No heartbeat from core client for 30 sec - exiting 19:49:34 (4364): No heartbeat from core client for 30 sec - exiting 19:49:35 (4364): No heartbeat from core client for 30 sec - exiting 19:49:36 (4364): No heartbeat from core client for 30 sec - exiting 19:49:37 (4364): No heartbeat from core client for 30 sec - exiting 19:49:38 (4364): No heartbeat from core client for 30 sec - exiting 19:49:39 (4364): No heartbeat from core client for 30 sec - exiting 19:49:40 (4364): No heartbeat from core client for 30 sec - exiting 19:49:41 (4364): No heartbeat from core client for 30 sec - exiting 19:49:42 (4364): No heartbeat from core client for 30 sec - exiting 19:49:44 (4364): No heartbeat from core client for 30 sec - exiting 19:49:45 (4364): No heartbeat from core client for 30 sec - exiting 19:49:46 (4364): No heartbeat from core client for 30 sec - exiting 19:49:47 (4364): No heartbeat from core client for 30 sec - exiting 19:49:48 (4364): No heartbeat from core client for 30 sec - exiting 19:49:49 (4364): No heartbeat from core client for 30 sec - exiting 19:49:50 (4364): No heartbeat from core client for 30 sec - exiting 19:49:51 (4364): No heartbeat from core client for 30 sec - exiting 19:49:52 (4364): No heartbeat from core client for 30 sec - exiting 19:49:53 (4364): No heartbeat from core client for 30 sec - exiting 19:49:54 (4364): No heartbeat from core client for 30 sec - exiting 19:49:56 (4364): No heartbeat from core client for 30 sec - exiting 19:49:57 (4364): No heartbeat from core client for 30 sec - exiting 19:49:58 (4364): No heartbeat from core client for 30 sec - exiting 19:49:59 (4364): No heartbeat from core client for 30 sec - exiting 19:50:00 (4364): No heartbeat from core client for 30 sec - exiting 19:50:01 (4364): No heartbeat from core client for 30 sec - exiting 19:50:02 (4364): No heartbeat from core client for 30 sec - exiting 19:50:03 (4364): No heartbeat from core client for 30 sec - exiting 19:50:04 (4364): No heartbeat from core client for 30 sec - exiting 19:50:05 (4364): No heartbeat from core client for 30 sec - exiting 19:50:06 (4364): No heartbeat from core client for 30 sec - exiting 19:50:08 (4364): No heartbeat from core client for 30 sec - exiting 19:50:09 (4364): No heartbeat from core client for 30 sec - exiting 19:50:10 (4364): No heartbeat from core client for 30 sec - exiting 19:50:11 (4364): No heartbeat from core client for 30 sec - exiting 19:50:12 (4364): No heartbeat from core client for 30 sec - exiting 19:50:13 (4364): No heartbeat from core client for 30 sec - exiting 19:50:14 (4364): No heartbeat from core client for 30 sec - exiting 19:50:15 (4364): No heartbeat from core client for 30 sec - exiting 19:50:16 (4364): No heartbeat from core client for 30 sec - exiting 19:50:17 (4364): No heartbeat from core client for 30 sec - exiting 19:50:18 (4364): No heartbeat from core client for 30 sec - exiting 19:50:20 (4364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:23 (2092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:01 (4328): No heartbeat from core client for 30 sec - exiting 20:11:02 (4328): No heartbeat from core client for 30 sec - exiting 20:11:03 (4328): No heartbeat from core client for 30 sec - exiting 20:11:04 (4328): No heartbeat from core client for 30 sec - exiting 20:11:05 (4328): No heartbeat from core client for 30 sec - exiting 20:11:06 (4328): No heartbeat from core client for 30 sec - exiting 20:11:07 (4328): No heartbeat from core client for 30 sec - exiting 20:11:09 (4328): No heartbeat from core client for 30 sec - exiting 20:11:10 (4328): No heartbeat from core client for 30 sec - exiting 20:11:11 (4328): No heartbeat from core client for 30 sec - exiting 20:11:12 (4328): No heartbeat from core client for 30 sec - exiting 20:11:13 (4328): No heartbeat from core client for 30 sec - exiting 20:11:14 (4328): No heartbeat from core client for 30 sec - exiting 20:11:15 (4328): No heartbeat from core client for 30 sec - exiting 20:11:16 (4328): No heartbeat from core client for 30 sec - exiting 20:11:17 (4328): No heartbeat from core client for 30 sec - exiting 20:11:18 (4328): No heartbeat from core client for 30 sec - exiting 20:11:19 (4328): No heartbeat from core client for 30 sec - exiting 20:11:20 (4328): No heartbeat from core client for 30 sec - exiting 20:11:22 (4328): No heartbeat from core client for 30 sec - exiting 20:11:23 (4328): No heartbeat from core client for 30 sec - exiting 20:11:24 (4328): No heartbeat from core client for 30 sec - exiting 20:11:25 (4328): No heartbeat from core client for 30 sec - exiting 20:11:26 (4328): No heartbeat from core client for 30 sec - exiting 20:11:27 (4328): No heartbeat from core client for 30 sec - exiting 20:11:28 (4328): No heartbeat from core client for 30 sec - exiting 20:11:29 (4328): No heartbeat from core client for 30 sec - exiting 20:11:30 (4328): No heartbeat from core client for 30 sec - exiting 20:11:31 (4328): No heartbeat from core client for 30 sec - exiting 20:11:32 (4328): No heartbeat from core client for 30 sec - exiting 20:11:34 (4328): No heartbeat from core client for 30 sec - exiting 20:11:35 (4328): No heartbeat from core client for 30 sec - exiting 20:11:36 (4328): No heartbeat from core client for 30 sec - exiting 20:11:37 (4328): No heartbeat from core client for 30 sec - exiting 20:11:38 (4328): No heartbeat from core client for 30 sec - exiting 20:11:39 (4328): No heartbeat from core client for 30 sec - exiting 20:11:40 (4328): No heartbeat from core client for 30 sec - exiting 20:11:41 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:46:31 (1456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:47:09 (4936): No heartbeat from core client for 30 sec - exiting 23:47:10 (4936): No heartbeat from core client for 30 sec - exiting 23:47:11 (4936): No heartbeat from core client for 30 sec - exiting 23:47:12 (4936): No heartbeat from core client for 30 sec - exiting 23:47:13 (4936): No heartbeat from core client for 30 sec - exiting 23:47:14 (4936): No heartbeat from core client for 30 sec - exiting 23:47:15 (4936): No heartbeat from core client for 30 sec - exiting 23:47:17 (4936): No heartbeat from core client for 30 sec - exiting 23:47:18 (4936): No heartbeat from core client for 30 sec - exiting 23:47:19 (4936): No heartbeat from core client for 30 sec - exiting 23:47:20 (4936): No heartbeat from core client for 30 sec - exiting 23:47:21 (4936): No heartbeat from core client for 30 sec - exiting 23:47:22 (4936): No heartbeat from core client for 30 sec - exiting 23:47:23 (4936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:53:00 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:29:42 (4768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5232, iMonCtr=1 Model crash detected, will try to restart... 05:29:50 (936): No heartbeat from core client for 30 sec - exiting 05:29:51 (936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:28:57 (4084): No heartbeat from core client for 30 sec - exiting 06:28:58 (4084): No heartbeat from core client for 30 sec - exiting 06:28:59 (4084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:29:00 (4084): No heartbeat from core client for 30 sec - exiting 09:17:12 (3204): No heartbeat from core client for 30 sec - exiting 09:17:14 (3204): No heartbeat from core client for 30 sec - exiting 09:17:15 (3204): No heartbeat from core client for 30 sec - exiting 09:17:16 (3204): No heartbeat from core client for 30 sec - exiting 09:17:17 (3204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:07:07 (4864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:15:11 (2100): No heartbeat from core client for 30 sec - exiting 16:15:13 (2100): No heartbeat from core client for 30 sec - exiting 16:15:14 (2100): No heartbeat from core client for 30 sec - exiting 16:15:15 (2100): No heartbeat from core client for 30 sec - exiting 16:15:16 (2100): No heartbeat from core client for 30 sec - exiting 16:15:17 (2100): No heartbeat from core client for 30 sec - exiting 16:15:18 (2100): No heartbeat from core client for 30 sec - exiting 16:15:19 (2100): No heartbeat from core client for 30 sec - exiting 16:15:20 (2100): No heartbeat from core client for 30 sec - exiting 16:15:21 (2100): No heartbeat from core client for 30 sec - exiting 16:15:22 (2100): No heartbeat from core client for 30 sec - exiting 16:15:23 (2100): No heartbeat from core client for 30 sec - exiting 16:15:25 (2100): No heartbeat from core client for 30 sec - exiting 16:15:26 (2100): No heartbeat from core client for 30 sec - exiting 16:15:27 (2100): No heartbeat from core client for 30 sec - exiting 16:15:28 (2100): No heartbeat from core client for 30 sec - exiting 16:15:29 (2100): No heartbeat from core client for 30 sec - exiting 16:15:30 (2100): No heartbeat from core client for 30 sec - exiting 16:15:31 (2100): No heartbeat from core client for 30 sec - exiting 16:15:32 (2100): No heartbeat from core client for 30 sec - exiting 16:15:33 (2100): No heartbeat from core client for 30 sec - exiting 16:15:34 (2100): No heartbeat from core client for 30 sec - exiting 16:15:35 (2100): No heartbeat from core client for 30 sec - exiting 16:15:37 (2100): No heartbeat from core client for 30 sec - exiting 16:15:38 (2100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4840, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=276, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 03:03:47 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:48 (4952): No heartbeat from core client for 30 sec - exiting 03:03:49 (4952): No heartbeat from core client for 30 sec - exiting 03:03:50 (4952): No heartbeat from core client for 30 sec - exiting 03:03:51 (4952): No heartbeat from core client for 30 sec - exiting 03:03:52 (4952): No heartbeat from core client for 30 sec - exiting 03:03:53 (4952): No heartbeat from core client for 30 sec - exiting 03:03:54 (4952): No heartbeat from core client for 30 sec - exiting 03:03:55 (4952): No heartbeat from core client for 30 sec - exiting 03:03:57 (4952): No heartbeat from core client for 30 sec - exiting 03:03:58 (4952): No heartbeat from core client for 30 sec - exiting 05:15:15 (3792): No heartbeat from core client for 30 sec - exiting 05:15:16 (3792): No heartbeat from core client for 30 sec - exiting 05:15:17 (3792): No heartbeat from core client for 30 sec - exiting 05:15:18 (3792): No heartbeat from core client for 30 sec - exiting 05:15:19 (3792): No heartbeat from core client for 30 sec - exiting 05:15:20 (3792): No heartbeat from core client for 30 sec - exiting 05:15:21 (3792): No heartbeat from core client for 30 sec - exiting 05:15:22 (3792): No heartbeat from core client for 30 sec - exiting 05:15:23 (3792): No heartbeat from core client for 30 sec - exiting 05:15:24 (3792): No heartbeat from core client for 30 sec - exiting 05:15:26 (3792): No heartbeat from core client for 30 sec - exiting 05:15:27 (3792): No heartbeat from core client for 30 sec - exiting 05:15:28 (3792): No heartbeat from core client for 30 sec - exiting 05:15:29 (3792): No heartbeat from core client for 30 sec - exiting 05:15:30 (3792): No heartbeat from core client for 30 sec - exiting 05:15:31 (3792): No heartbeat from core client for 30 sec - exiting 05:15:32 (3792): No heartbeat from core client for 30 sec - exiting 05:15:33 (3792): No heartbeat from core client for 30 sec - exiting 05:15:34 (3792): No heartbeat from core client for 30 sec - exiting 05:15:35 (3792): No heartbeat from core client for 30 sec - exiting 05:15:36 (3792): No heartbeat from core client for 30 sec - exiting 05:15:38 (3792): No heartbeat from core client for 30 sec - exiting 05:15:39 (3792): No heartbeat from core client for 30 sec - exiting 05:15:40 (3792): No heartbeat from core client for 30 sec - exiting 05:15:41 (3792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:57 (5064): No heartbeat from core client for 30 sec - exiting 05:28:59 (5064): No heartbeat from core client for 30 sec - exiting 05:29:00 (5064): No heartbeat from core client for 30 sec - exiting 05:29:01 (5064): No heartbeat from core client for 30 sec - exiting 05:29:02 (5064): No heartbeat from core client for 30 sec - exiting 05:29:03 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:49:21 (892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4844, iMonCtr=1 Model crash detected, will try to restart... 05:13:19 (4820): No heartbeat from core client for 30 sec - exiting 05:13:21 (4820): No heartbeat from core client for 30 sec - exiting 05:13:22 (4820): No heartbeat from core client for 30 sec - exiting 05:13:23 (4820): No heartbeat from core client for 30 sec - exiting 05:13:24 (4820): No heartbeat from core client for 30 sec - exiting 05:13:25 (4820): No heartbeat from core client for 30 sec - exiting 05:13:26 (4820): No heartbeat from core client for 30 sec - exiting 05:13:27 (4820): No heartbeat from core client for 30 sec - exiting 05:13:28 (4820): No heartbeat from core client for 30 sec - exiting 05:13:29 (4820): No heartbeat from core client for 30 sec - exiting 05:13:30 (4820): No heartbeat from core client for 30 sec - exiting 05:13:31 (4820): No heartbeat from core client for 30 sec - exiting 05:13:33 (4820): No heartbeat from core client for 30 sec - exiting 05:13:34 (4820): No heartbeat from core client for 30 sec - exiting 05:13:35 (4820): No heartbeat from core client for 30 sec - exiting 05:13:36 (4820): No heartbeat from core client for 30 sec - exiting 05:13:37 (4820): No heartbeat from core client for 30 sec - exiting 05:13:38 (4820): No heartbeat from core client for 30 sec - exiting 05:13:39 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:28:16 (4508): No heartbeat from core client for 30 sec - exiting 05:28:17 (4508): No heartbeat from core client for 30 sec - exiting 05:28:18 (4508): No heartbeat from core client for 30 sec - exiting 05:28:19 (4508): No heartbeat from core client for 30 sec - exiting 05:28:20 (4508): No heartbeat from core client for 30 sec - exiting 05:28:21 (4508): No heartbeat from core client for 30 sec - exiting 05:28:22 (4508): No heartbeat from core client for 30 sec - exiting 05:28:23 (4508): No heartbeat from core client for 30 sec - exiting 05:28:24 (4508): No heartbeat from core client for 30 sec - exiting 05:28:25 (4508): No heartbeat from core client for 30 sec - exiting 05:28:27 (4508): No heartbeat from core client for 30 sec - exiting 05:28:28 (4508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:35:26 (4680): No heartbeat from core client for 30 sec - exiting 17:35:28 (4680): No heartbeat from core client for 30 sec - exiting 17:35:29 (4680): No heartbeat from core client for 30 sec - exiting 17:35:30 (4680): No heartbeat from core client for 30 sec - exiting 17:35:31 (4680): No heartbeat from core client for 30 sec - exiting 17:35:32 (4680): No heartbeat from core client for 30 sec - exiting 17:35:33 (4680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:07:22 (4972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:23:16 (2212): No heartbeat from core client for 30 sec - exiting 05:23:17 (2212): No heartbeat from core client for 30 sec - exiting 05:23:18 (2212): No heartbeat from core client for 30 sec - exiting 05:23:19 (2212): No heartbeat from core client for 30 sec - exiting 05:23:20 (2212): No heartbeat from core client for 30 sec - exiting 05:23:21 (2212): No heartbeat from core client for 30 sec - exiting 05:23:22 (2212): No heartbeat from core client for 30 sec - exiting 05:23:23 (2212): No heartbeat from core client for 30 sec - exiting 05:23:24 (2212): No heartbeat from core client for 30 sec - exiting 05:23:26 (2212): No heartbeat from core client for 30 sec - exiting 05:23:27 (2212): No heartbeat from core client for 30 sec - exiting 05:23:28 (2212): No heartbeat from core client for 30 sec - exiting 05:23:29 (2212): No heartbeat from core client for 30 sec - exiting 05:23:30 (2212): No heartbeat from core client for 30 sec - exiting 05:23:31 (2212): No heartbeat from core client for 30 sec - exiting 05:23:32 (2212): No heartbeat from core client for 30 sec - exiting 05:23:33 (2212): No heartbeat from core client for 30 sec - exiting 05:23:34 (2212): No heartbeat from core client for 30 sec - exiting 05:23:35 (2212): No heartbeat from core client for 30 sec - exiting 05:23:36 (2212): No heartbeat from core client for 30 sec - exiting 05:23:38 (2212): No heartbeat from core client for 30 sec - exiting 05:23:39 (2212): No heartbeat from core client for 30 sec - exiting 05:23:40 (2212): No heartbeat from core client for 30 sec - exiting 05:23:41 (2212): No heartbeat from core client for 30 sec - exiting 05:23:42 (2212): No heartbeat from core client for 30 sec - exiting 05:23:43 (2212): No heartbeat from core client for 30 sec - exiting 05:23:44 (2212): No heartbeat from core client for 30 sec - exiting 05:23:45 (2212): No heartbeat from core client for 30 sec - exiting 05:23:46 (2212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:29:50 (3112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:35:16 (4328): No heartbeat from core client for 30 sec - exiting 22:35:17 (4328): No heartbeat from core client for 30 sec - exiting 22:35:18 (4328): No heartbeat from core client for 30 sec - exiting 22:35:19 (4328): No heartbeat from core client for 30 sec - exiting 22:35:20 (4328): No heartbeat from core client for 30 sec - exiting 22:35:21 (4328): No heartbeat from core client for 30 sec - exiting 22:35:22 (4328): No heartbeat from core client for 30 sec - exiting 22:35:24 (4328): No heartbeat from core client for 30 sec - exiting 22:35:25 (4328): No heartbeat from core client for 30 sec - exiting 22:35:26 (4328): No heartbeat from core client for 30 sec - exiting 22:35:27 (4328): No heartbeat from core client for 30 sec - exiting 22:35:28 (4328): No heartbeat from core client for 30 sec - exiting 22:35:29 (4328): No heartbeat from core client for 30 sec - exiting 22:35:30 (4328): No heartbeat from core client for 30 sec - exiting 22:35:31 (4328): No heartbeat from core client for 30 sec - exiting 22:35:32 (4328): No heartbeat from core client for 30 sec - exiting 22:35:33 (4328): No heartbeat from core client for 30 sec - exiting 22:35:35 (4328): No heartbeat from core client for 30 sec - exiting 22:35:36 (4328): No heartbeat from core client for 30 sec - exiting 22:35:37 (4328): No heartbeat from core client for 30 sec - exiting 22:35:38 (4328): No heartbeat from core client for 30 sec - exiting 22:35:39 (4328): No heartbeat from core client for 30 sec - exiting 22:35:40 (4328): No heartbeat from core client for 30 sec - exiting 22:35:41 (4328): No heartbeat from core client for 30 sec - exiting 22:35:42 (4328): No heartbeat from core client for 30 sec - exiting 22:35:43 (4328): No heartbeat from core client for 30 sec - exiting 22:35:44 (4328): No heartbeat from core client for 30 sec - exiting 22:35:45 (4328): No heartbeat from core client for 30 sec - exiting 22:35:47 (4328): No heartbeat from core client for 30 sec - exiting 22:35:48 (4328): No heartbeat from core client for 30 sec - exiting 22:35:49 (4328): No heartbeat from core client for 30 sec - exiting 22:35:50 (4328): No heartbeat from core client for 30 sec - exiting 22:35:51 (4328): No heartbeat from core client for 30 sec - exiting 22:35:52 (4328): No heartbeat from core client for 30 sec - exiting 22:35:53 (4328): No heartbeat from core client for 30 sec - exiting 22:35:54 (4328): No heartbeat from core client for 30 sec - exiting 22:35:55 (4328): No heartbeat from core client for 30 sec - exiting 22:35:56 (4328): No heartbeat from core client for 30 sec - exiting 22:35:57 (4328): No heartbeat from core client for 30 sec - exiting 22:35:59 (4328): No heartbeat from core client for 30 sec - exiting 22:36:00 (4328): No heartbeat from core client for 30 sec - exiting 22:36:01 (4328): No heartbeat from core client for 30 sec - exiting 22:36:02 (4328): No heartbeat from core client for 30 sec - exiting 22:36:03 (4328): No heartbeat from core client for 30 sec - exiting 22:36:04 (4328): No heartbeat from core client for 30 sec - exiting 22:36:05 (4328): No heartbeat from core client for 30 sec - exiting 22:36:06 (4328): No heartbeat from core client for 30 sec - exiting 22:36:07 (4328): No heartbeat from core client for 30 sec - exiting 22:36:08 (4328): No heartbeat from core client for 30 sec - exiting 22:36:09 (4328): No heartbeat from core client for 30 sec - exiting 22:36:11 (4328): No heartbeat from core client for 30 sec - exiting 22:36:12 (4328): No heartbeat from core client for 30 sec - exiting 22:36:13 (4328): No heartbeat from core client for 30 sec - exiting 22:36:14 (4328): No heartbeat from core client for 30 sec - exiting 22:36:15 (4328): No heartbeat from core client for 30 sec - exiting 22:36:16 (4328): No heartbeat from core client for 30 sec - exiting 22:36:17 (4328): No heartbeat from core client for 30 sec - exiting 22:36:18 (4328): No heartbeat from core client for 30 sec - exiting 22:36:19 (4328): No heartbeat from core client for 30 sec - exiting 22:36:20 (4328): No heartbeat from core client for 30 sec - exiting 22:36:21 (4328): No heartbeat from core client for 30 sec - exiting 22:36:23 (4328): No heartbeat from core client for 30 sec - exiting 22:36:24 (4328): No heartbeat from core client for 30 sec - exiting 22:36:25 (4328): No heartbeat from core client for 30 sec - exiting 22:36:26 (4328): No heartbeat from core client for 30 sec - exiting 22:36:27 (4328): No heartbeat from core client for 30 sec - exiting 22:36:28 (4328): No heartbeat from core client for 30 sec - exiting 22:36:29 (4328): No heartbeat from core client for 30 sec - exiting 22:36:30 (4328): No heartbeat from core client for 30 sec - exiting 22:36:31 (4328): No heartbeat from core client for 30 sec - exiting 22:36:32 (4328): No heartbeat from core client for 30 sec - exiting 22:36:33 (4328): No heartbeat from core client for 30 sec - exiting 22:36:35 (4328): No heartbeat from core client for 30 sec - exiting 22:36:36 (4328): No heartbeat from core client for 30 sec - exiting 22:36:37 (4328): No heartbeat from core client for 30 sec - exiting 22:36:38 (4328): No heartbeat from core client for 30 sec - exiting 22:36:39 (4328): No heartbeat from core client for 30 sec - exiting 22:36:40 (4328): No heartbeat from core client for 30 sec - exiting 22:36:41 (4328): No heartbeat from core client for 30 sec - exiting 22:36:42 (4328): No heartbeat from core client for 30 sec - exiting 22:36:43 (4328): No heartbeat from core client for 30 sec - exiting 22:36:44 (4328): No heartbeat from core client for 30 sec - exiting 22:36:45 (4328): No heartbeat from core client for 30 sec - exiting 22:36:47 (4328): No heartbeat from core client for 30 sec - exiting 22:36:48 (4328): No heartbeat from core client for 30 sec - exiting 22:36:49 (4328): No heartbeat from core client for 30 sec - exiting 22:36:50 (4328): No heartbeat from core client for 30 sec - exiting 22:36:51 (4328): No heartbeat from core client for 30 sec - exiting 22:36:52 (4328): No heartbeat from core client for 30 sec - exiting 22:36:53 (4328): No heartbeat from core client for 30 sec - exiting 22:36:54 (4328): No heartbeat from core client for 30 sec - exiting 22:36:55 (4328): No heartbeat from core client for 30 sec - exiting 22:36:56 (4328): No heartbeat from core client for 30 sec - exiting 22:36:57 (4328): No heartbeat from core client for 30 sec - exiting 22:36:59 (4328): No heartbeat from core client for 30 sec - exiting 22:37:00 (4328): No heartbeat from core client for 30 sec - exiting 22:37:01 (4328): No heartbeat from core client for 30 sec - exiting 22:37:02 (4328): No heartbeat from core client for 30 sec - exiting 22:37:03 (4328): No heartbeat from core client for 30 sec - exiting 22:37:04 (4328): No heartbeat from core client for 30 sec - exiting 22:37:05 (4328): No heartbeat from core client for 30 sec - exiting 22:37:06 (4328): No heartbeat from core client for 30 sec - exiting 22:37:07 (4328): No heartbeat from core client for 30 sec - exiting 22:37:08 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:38:01 (4980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:22:40 (4784): No heartbeat from core client for 30 sec - exiting 05:22:42 (4784): No heartbeat from core client for 30 sec - exiting 05:22:43 (4784): No heartbeat from core client for 30 sec - exiting 05:22:44 (4784): No heartbeat from core client for 30 sec - exiting 05:22:45 (4784): No heartbeat from core client for 30 sec - exiting 05:22:46 (4784): No heartbeat from core client for 30 sec - exiting 05:22:47 (4784): No heartbeat from core client for 30 sec - exiting 05:22:48 (4784): No heartbeat from core client for 30 sec - exiting 05:22:49 (4784): No heartbeat from core client for 30 sec - exiting 05:22:50 (4784): No heartbeat from core client for 30 sec - exiting 05:22:51 (4784): No heartbeat from core client for 30 sec - exiting 05:22:52 (4784): No heartbeat from core client for 30 sec - exiting 05:22:54 (4784): No heartbeat from core client for 30 sec - exiting 05:22:55 (4784): No heartbeat from core client for 30 sec - exiting 05:22:56 (4784): No heartbeat from core client for 30 sec - exiting 05:22:57 (4784): No heartbeat from core client for 30 sec - exiting 05:22:58 (4784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:51:17 (5092): No heartbeat from core client for 30 sec - exiting 19:51:18 (5092): No heartbeat from core client for 30 sec - exiting 19:51:19 (5092): No heartbeat from core client for 30 sec - exiting 19:51:20 (5092): No heartbeat from core client for 30 sec - exiting 19:51:21 (5092): No heartbeat from core client for 30 sec - exiting 19:51:22 (5092): No heartbeat from core client for 30 sec - exiting 19:51:23 (5092): No heartbeat from core client for 30 sec - exiting 19:51:24 (5092): No heartbeat from core client for 30 sec - exiting 19:51:25 (5092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:58:00 (4636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:56 (4968): No heartbeat from core client for 30 sec - exiting 05:22:57 (4968): No heartbeat from core client for 30 sec - exiting 05:22:59 (4968): No heartbeat from core client for 30 sec - exiting 05:23:00 (4968): No heartbeat from core client for 30 sec - exiting 05:23:01 (4968): No heartbeat from core client for 30 sec - exiting 05:23:02 (4968): No heartbeat from core client for 30 sec - exiting 05:23:03 (4968): No heartbeat from core client for 30 sec - exiting 05:23:04 (4968): No heartbeat from core client for 30 sec - exiting 05:23:05 (4968): No heartbeat from core client for 30 sec - exiting 05:23:06 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Jun 2013 12:29:50	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	518,400	893,830	1.7242
10 Jun 2013 17:42:04	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	492,480	849,218	1.7244
10 Jun 2013 03:00:24	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	466,560	803,883	1.7230
09 Jun 2013 08:34:36	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	440,640	758,733	1.7219
08 Jun 2013 15:06:23	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	414,720	714,110	1.7219
08 Jun 2013 01:20:25	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	388,800	669,617	1.7223
07 Jun 2013 11:01:26	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	362,880	625,647	1.7241
06 Jun 2013 21:29:49	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	336,960	580,080	1.7215
06 Jun 2013 08:26:10	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	311,040	535,775	1.7225
05 Jun 2013 18:56:59	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	285,120	490,384	1.7199
05 Jun 2013 03:58:22	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	259,200	444,401	1.7145
04 Jun 2013 14:32:26	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	233,280	398,791	1.7095
03 Jun 2013 23:59:31	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	207,360	352,735	1.7011
03 Jun 2013 09:39:17	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	181,440	307,589	1.6953
02 Jun 2013 19:31:24	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	155,520	262,056	1.6850
02 Jun 2013 03:06:41	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	129,600	218,050	1.6825
01 Jun 2013 14:11:31	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	103,680	174,736	1.6853
31 May 2013 22:29:55	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	77,760	130,991	1.6846
31 May 2013 08:11:03	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	51,840	87,369	1.6854
30 May 2013 17:51:06	1230144	15804975	hadcm3n_o0eh_2020_40_008376112_0	25,920	43,489	1.6778