Task 15893242

Name	hadcm3n_38bl_1940_40_008265770_4
Workunit	8420894
Created	18 Jul 2013, 10:51:19 UTC
Sent	18 Jul 2013, 21:00:17 UTC
Report deadline	18 Oct 2013, 4:27:28 UTC
Received	24 Sep 2013, 20:26:04 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1196485
Run time	15 days 5 hours 21 min 43 sec
CPU time	14 days 5 hours 59 min 30 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.40 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:19:05 (5076): No heartbeat from core client for 30 sec - exiting 07:19:06 (5076): No heartbeat from core client for 30 sec - exiting 07:19:07 (5076): No heartbeat from core client for 30 sec - exiting 07:19:08 (5076): No heartbeat from core client for 30 sec - exiting 07:19:09 (5076): No heartbeat from core client for 30 sec - exiting 07:19:10 (5076): No heartbeat from core client for 30 sec - exiting 07:19:11 (5076): No heartbeat from core client for 30 sec - exiting 07:19:12 (5076): No heartbeat from core client for 30 sec - exiting 07:19:13 (5076): No heartbeat from core client for 30 sec - exiting 07:19:14 (5076): No heartbeat from core client for 30 sec - exiting 07:19:15 (5076): No heartbeat from core client for 30 sec - exiting 07:19:16 (5076): No heartbeat from core client for 30 sec - exiting 07:19:17 (5076): No heartbeat from core client for 30 sec - exiting 07:19:18 (5076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:47:36 (9848): No heartbeat from core client for 30 sec - exiting 21:47:37 (9848): No heartbeat from core client for 30 sec - exiting 21:47:38 (9848): No heartbeat from core client for 30 sec - exiting 21:47:39 (9848): No heartbeat from core client for 30 sec - exiting 21:47:40 (9848): No heartbeat from core client for 30 sec - exiting 21:47:41 (9848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:43:34 (8044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:04:04 (9560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:34:20 (6720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:43:46 (7176): No heartbeat from core client for 30 sec - exiting 08:43:48 (7176): No heartbeat from core client for 30 sec - exiting 08:43:49 (7176): No heartbeat from core client for 30 sec - exiting 08:43:50 (7176): No heartbeat from core client for 30 sec - exiting 08:43:51 (7176): No heartbeat from core client for 30 sec - exiting 08:43:52 (7176): No heartbeat from core client for 30 sec - exiting 08:43:53 (7176): No heartbeat from core client for 30 sec - exiting 08:43:54 (7176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:33:08 (4652): No heartbeat from core client for 30 sec - exiting 07:33:10 (4652): No heartbeat from core client for 30 sec - exiting 07:33:11 (4652): No heartbeat from core client for 30 sec - exiting 07:33:12 (4652): No heartbeat from core client for 30 sec - exiting 07:33:13 (4652): No heartbeat from core client for 30 sec - exiting 07:33:14 (4652): No heartbeat from core client for 30 sec - exiting 07:33:15 (4652): No heartbeat from core client for 30 sec - exiting 07:33:16 (4652): No heartbeat from core client for 30 sec - exiting 07:33:17 (4652): No heartbeat from core client for 30 sec - exiting 07:33:18 (4652): No heartbeat from core client for 30 sec - exiting 07:33:19 (4652): No heartbeat from core client for 30 sec - exiting 07:33:20 (4652): No heartbeat from core client for 30 sec - exiting 07:33:22 (4652): No heartbeat from core client for 30 sec - exiting 07:33:23 (4652): No heartbeat from core client for 30 sec - exiting 07:33:24 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:33:25 (4652): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:32:41 (2728): No heartbeat from core client for 30 sec - exiting 07:32:42 (2728): No heartbeat from core client for 30 sec - exiting 07:32:43 (2728): No heartbeat from core client for 30 sec - exiting 07:32:44 (2728): No heartbeat from core client for 30 sec - exiting 07:32:45 (2728): No heartbeat from core client for 30 sec - exiting 07:32:46 (2728): No heartbeat from core client for 30 sec - exiting 07:32:47 (2728): No heartbeat from core client for 30 sec - exiting 07:32:48 (2728): No heartbeat from core client for 30 sec - exiting 07:32:49 (2728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:59:33 (10052): No heartbeat from core client for 30 sec - exiting 07:59:34 (10052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:55:31 (6024): No heartbeat from core client for 30 sec - exiting 08:55:33 (6024): No heartbeat from core client for 30 sec - exiting 08:55:34 (6024): No heartbeat from core client for 30 sec - exiting 08:55:35 (6024): No heartbeat from core client for 30 sec - exiting 08:55:36 (6024): No heartbeat from core client for 30 sec - exiting 08:55:37 (6024): No heartbeat from core client for 30 sec - exiting 08:55:38 (6024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:47:09 (4888): No heartbeat from core client for 30 sec - exiting 19:47:11 (4888): No heartbeat from core client for 30 sec - exiting 19:47:12 (4888): No heartbeat from core client for 30 sec - exiting 19:47:13 (4888): No heartbeat from core client for 30 sec - exiting 19:47:14 (4888): No heartbeat from core client for 30 sec - exiting 19:47:15 (4888): No heartbeat from core client for 30 sec - exiting 19:47:16 (4888): No heartbeat from core client for 30 sec - exiting 19:47:17 (4888): No heartbeat from core client for 30 sec - exiting 19:47:18 (4888): No heartbeat from core client for 30 sec - exiting 19:47:19 (4888): No heartbeat from core client for 30 sec - exiting 19:47:20 (4888): No heartbeat from core client for 30 sec - exiting 19:47:21 (4888): No heartbeat from core client for 30 sec - exiting 19:47:22 (4888): No heartbeat from core client for 30 sec - exiting 19:47:23 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:55:39 (8984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:48:55 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:50:09 (3076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:23:52 (6628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:53:19 (6636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:14:14 (6304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:45:13 (3108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:39:53 (6516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1172, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:50:46 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:36:21 (4292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:52:36 (7848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:31:44 (3172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:59:59 (9172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:14:20 (1716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:08:00 (2588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:40:13 (4792): No heartbeat from core client for 30 sec - exiting 07:40:15 (4792): No heartbeat from core client for 30 sec - exiting 07:40:16 (4792): No heartbeat from core client for 30 sec - exiting 07:40:17 (4792): No heartbeat from core client for 30 sec - exiting 07:40:18 (4792): No heartbeat from core client for 30 sec - exiting 07:40:19 (4792): No heartbeat from core client for 30 sec - exiting 07:40:20 (4792): No heartbeat from core client for 30 sec - exiting 07:40:21 (4792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:46:05 (2716): No heartbeat from core client for 30 sec - exiting 07:46:06 (2716): No heartbeat from core client for 30 sec - exiting 07:46:08 (2716): No heartbeat from core client for 30 sec - exiting 07:46:09 (2716): No heartbeat from core client for 30 sec - exiting 07:46:10 (2716): No heartbeat from core client for 30 sec - exiting 07:46:11 (2716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:31:57 (3180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:31:58 (3180): No heartbeat from core client for 30 sec - exiting 08:31:59 (3180): No heartbeat from core client for 30 sec - exiting 08:32:00 (3180): No heartbeat from core client for 30 sec - exiting 08:32:01 (3180): No heartbeat from core client for 30 sec - exiting 08:32:02 (3180): No heartbeat from core client for 30 sec - exiting 08:32:03 (3180): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:43:57 (6420): No heartbeat from core client for 30 sec - exiting 21:43:58 (6420): No heartbeat from core client for 30 sec - exiting 21:43:59 (6420): No heartbeat from core client for 30 sec - exiting 21:44:00 (6420): No heartbeat from core client for 30 sec - exiting 21:44:01 (6420): No heartbeat from core client for 30 sec - exiting 21:44:02 (6420): No heartbeat from core client for 30 sec - exiting 21:44:03 (6420): No heartbeat from core client for 30 sec - exiting 21:44:04 (6420): No heartbeat from core client for 30 sec - exiting 21:44:05 (6420): No heartbeat from core client for 30 sec - exiting 21:44:06 (6420): No heartbeat from core client for 30 sec - exiting 21:44:07 (6420): No heartbeat from core client for 30 sec - exiting 21:44:08 (6420): No heartbeat from core client for 30 sec - exiting 21:44:09 (6420): No heartbeat from core client for 30 sec - exiting 21:44:10 (6420): No heartbeat from core client for 30 sec - exiting 21:44:11 (6420): No heartbeat from core client for 30 sec - exiting 21:44:12 (6420): No heartbeat from core client for 30 sec - exiting 21:44:13 (6420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:28:05 (8492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:55:11 (8700): No heartbeat from core client for 30 sec - exiting 07:55:12 (8700): No heartbeat from core client for 30 sec - exiting 07:55:13 (8700): No heartbeat from core client for 30 sec - exiting 07:55:14 (8700): No heartbeat from core client for 30 sec - exiting 07:55:15 (8700): No heartbeat from core client for 30 sec - exiting 07:55:16 (8700): No heartbeat from core client for 30 sec - exiting 07:55:17 (8700): No heartbeat from core client for 30 sec - exiting 07:55:18 (8700): No heartbeat from core client for 30 sec - exiting 07:55:19 (8700): No heartbeat from core client for 30 sec - exiting 07:55:20 (8700): No heartbeat from core client for 30 sec - exiting 07:55:21 (8700): No heartbeat from core client for 30 sec - exiting 07:55:22 (8700): No heartbeat from core client for 30 sec - exiting 07:55:23 (8700): No heartbeat from core client for 30 sec - exiting 07:55:24 (8700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:09:24 (5456): No heartbeat from core client for 30 sec - exiting 09:09:25 (5456): No heartbeat from core client for 30 sec - exiting 09:09:26 (5456): No heartbeat from core client for 30 sec - exiting 09:09:27 (5456): No heartbeat from core client for 30 sec - exiting 09:09:28 (5456): No heartbeat from core client for 30 sec - exiting 09:09:29 (5456): No heartbeat from core client for 30 sec - exiting 09:09:30 (5456): No heartbeat from core client for 30 sec - exiting 09:09:31 (5456): No heartbeat from core client for 30 sec - exiting 09:09:32 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:31:06 (6364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:18:43 (1484): No heartbeat from core client for 30 sec - exiting 08:18:44 (1484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:27:34 (6320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:11:37 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:59:03 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:43:28 (6064): No heartbeat from core client for 30 sec - exiting 08:43:29 (6064): No heartbeat from core client for 30 sec - exiting 08:43:30 (6064): No heartbeat from core client for 30 sec - exiting 08:43:31 (6064): No heartbeat from core client for 30 sec - exiting 08:43:32 (6064): No heartbeat from core client for 30 sec - exiting 08:43:33 (6064): No heartbeat from core client for 30 sec - exiting 08:43:34 (6064): No heartbeat from core client for 30 sec - exiting 08:43:35 (6064): No heartbeat from core client for 30 sec - exiting 08:43:36 (6064): No heartbeat from core client for 30 sec - exiting 08:43:37 (6064): No heartbeat from core client for 30 sec - exiting 08:43:38 (6064): No heartbeat from core client for 30 sec - exiting 08:43:39 (6064): No heartbeat from core client for 30 sec - exiting 08:43:40 (6064): No heartbeat from core client for 30 sec - exiting 08:43:41 (6064): No heartbeat from core client for 30 sec - exiting 08:43:42 (6064): No heartbeat from core client for 30 sec - exiting 08:43:43 (6064): No heartbeat from core client for 30 sec - exiting 08:43:44 (6064): No heartbeat from core client for 30 sec - exiting 08:43:45 (6064): No heartbeat from core client for 30 sec - exiting 08:43:46 (6064): No heartbeat from core client for 30 sec - exiting 08:43:47 (6064): No heartbeat from core client for 30 sec - exiting 08:43:48 (6064): No heartbeat from core client for 30 sec - exiting 08:43:49 (6064): No heartbeat from core client for 30 sec - exiting 08:43:50 (6064): No heartbeat from core client for 30 sec - exiting 08:43:51 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5356, iMonCtr=1 Model crash detected, will try to restart... 07:37:00 (5432): No heartbeat from core client for 30 sec - exiting 07:37:01 (5432): No heartbeat from core client for 30 sec - exiting 07:37:03 (5432): No heartbeat from core client for 30 sec - exiting 07:37:04 (5432): No heartbeat from core client for 30 sec - exiting 07:37:05 (5432): No heartbeat from core client for 30 sec - exiting 07:37:06 (5432): No heartbeat from core client for 30 sec - exiting 07:37:07 (5432): No heartbeat from core client for 30 sec - exiting 07:37:08 (5432): No heartbeat from core client for 30 sec - exiting 07:37:09 (5432): No heartbeat from core client for 30 sec - exiting 07:37:10 (5432): No heartbeat from core client for 30 sec - exiting 07:37:11 (5432): No heartbeat from core client for 30 sec - exiting 07:37:12 (5432): No heartbeat from core client for 30 sec - exiting 07:37:13 (5432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:50:44 (3472): No heartbeat from core client for 30 sec - exiting 07:50:46 (3472): No heartbeat from core client for 30 sec - exiting 07:50:47 (3472): No heartbeat from core client for 30 sec - exiting 07:50:48 (3472): No heartbeat from core client for 30 sec - exiting 07:50:49 (3472): No heartbeat from core client for 30 sec - exiting 07:50:50 (3472): No heartbeat from core client for 30 sec - exiting 07:50:51 (3472): No heartbeat from core client for 30 sec - exiting 07:50:52 (3472): No heartbeat from core client for 30 sec - exiting 07:50:53 (3472): No heartbeat from core client for 30 sec - exiting 07:50:54 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6212, iMonCtr=1 Model crash detected, will try to restart... 08:07:26 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3576, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6704, iMonCtr=1 Model crash detected, will try to restart... 07:36:18 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Sep 2013 09:23:54	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	1,036,800	1,231,167	1.1875
23 Sep 2013 17:11:05	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	1,010,880	1,192,803	1.1800
20 Sep 2013 14:06:32	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	984,960	1,154,183	1.1718
18 Sep 2013 18:38:05	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	959,040	1,115,658	1.1633
17 Sep 2013 16:47:46	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	933,120	1,078,058	1.1553
16 Sep 2013 15:38:50	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	907,200	1,043,298	1.1500
13 Sep 2013 14:17:19	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	881,280	1,007,952	1.1437
11 Sep 2013 20:34:34	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	855,360	978,713	1.1442
10 Sep 2013 21:02:39	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	829,440	949,548	1.1448
09 Sep 2013 20:39:17	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	803,520	920,207	1.1452
08 Sep 2013 21:35:34	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	777,600	890,221	1.1448
06 Sep 2013 17:24:59	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	751,680	860,811	1.1452
05 Sep 2013 17:51:43	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	725,760	832,263	1.1467
04 Sep 2013 18:44:44	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	699,840	801,690	1.1455
03 Sep 2013 15:58:27	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	673,920	770,840	1.1438
29 Aug 2013 21:05:57	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	648,000	740,871	1.1433
28 Aug 2013 21:01:34	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	622,080	710,721	1.1425
27 Aug 2013 16:53:18	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	596,160	680,822	1.1420
26 Aug 2013 17:16:13	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	570,240	651,047	1.1417
23 Aug 2013 12:26:48	1196485	15893242	hadcm3n_38bl_1940_40_008265770_4	544,320	620,963	1.1408