Task 13638488

Name	hadcm3n_o053_1980_40_007535804_2
Workunit	7733036
Created	16 Nov 2011, 6:50:37 UTC
Sent	16 Nov 2011, 6:50:52 UTC
Report deadline	15 Feb 2012, 14:18:03 UTC
Received	27 Dec 2011, 14:55:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1123512
Run time	16 days 9 hours 6 min 26 sec
CPU time	13 days 14 hours 32 min 26 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.13.12</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 18:17:22 (1132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:23:40 (8684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:01:23 (10004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:43:55 (8148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:43:56 (8148): No heartbeat from core client for 30 sec - exiting 03:43:57 (8148): No heartbeat from core client for 30 sec - exiting 03:43:58 (8148): No heartbeat from core client for 30 sec - exiting 03:43:59 (8148): No heartbeat from core client for 30 sec - exiting 03:44:00 (8148): No heartbeat from core client for 30 sec - exiting 03:44:01 (8148): No heartbeat from core client for 30 sec - exiting 03:44:20 (7496): Can't acquire lockfile (32) - waiting 35s 04:59:31 (7496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:59:32 (7496): No heartbeat from core client for 30 sec - exiting 05:08:36 (6572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:27 (6888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:28 (6888): No heartbeat from core client for 30 sec - exiting 07:15:29 (6888): No heartbeat from core client for 30 sec - exiting 07:15:30 (6888): No heartbeat from core client for 30 sec - exiting 07:15:31 (6888): No heartbeat from core client for 30 sec - exiting 07:15:32 (6888): No heartbeat from core client for 30 sec - exiting 07:15:33 (6888): No heartbeat from core client for 30 sec - exiting 07:15:34 (6888): No heartbeat from core client for 30 sec - exiting 07:15:35 (6888): No heartbeat from core client for 30 sec - exiting 08:03:47 (3440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:03:48 (3440): No heartbeat from core client for 30 sec - exiting 08:06:53 (7988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:06:54 (7988): No heartbeat from core client for 30 sec - exiting 08:12:56 (8948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:12:57 (8948): No heartbeat from core client for 30 sec - exiting 08:12:58 (8948): No heartbeat from core client for 30 sec - exiting 08:12:59 (8948): No heartbeat from core client for 30 sec - exiting 08:13:00 (8948): No heartbeat from core client for 30 sec - exiting 08:13:01 (8948): No heartbeat from core client for 30 sec - exiting 08:13:02 (8948): No heartbeat from core client for 30 sec - exiting 08:13:03 (8948): No heartbeat from core client for 30 sec - exiting 08:13:04 (8948): No heartbeat from core client for 30 sec - exiting 08:13:05 (8948): No heartbeat from core client for 30 sec - exiting 08:13:06 (8948): No heartbeat from core client for 30 sec - exiting 08:58:28 (1524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:58:29 (1524): No heartbeat from core client for 30 sec - exiting 09:01:34 (4824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:01:35 (4824): No heartbeat from core client for 30 sec - exiting 09:01:58 (9916): Can't acquire lockfile (32) - waiting 35s 09:04:36 (9916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:37 (9916): No heartbeat from core client for 30 sec - exiting 09:04:38 (9916): No heartbeat from core client for 30 sec - exiting 09:04:39 (9916): No heartbeat from core client for 30 sec - exiting 09:04:40 (9916): No heartbeat from core client for 30 sec - exiting 09:04:41 (9916): No heartbeat from core client for 30 sec - exiting 10:26:07 (8736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:08 (8736): No heartbeat from core client for 30 sec - exiting 10:26:09 (8736): No heartbeat from core client for 30 sec - exiting 10:26:10 (8736): No heartbeat from core client for 30 sec - exiting 10:26:11 (8736): No heartbeat from core client for 30 sec - exiting 10:26:12 (8736): No heartbeat from core client for 30 sec - exiting 12:15:15 (10032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:16 (10032): No heartbeat from core client for 30 sec - exiting 12:15:17 (10032): No heartbeat from core client for 30 sec - exiting 12:15:18 (10032): No heartbeat from core client for 30 sec - exiting 12:18:19 (6412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:42:29 (10196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:42:30 (10196): No heartbeat from core client for 30 sec - exiting 12:42:31 (10196): No heartbeat from core client for 30 sec - exiting 12:42:32 (10196): No heartbeat from core client for 30 sec - exiting 12:42:33 (10196): No heartbeat from core client for 30 sec - exiting 14:13:19 (7304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:13:20 (7304): No heartbeat from core client for 30 sec - exiting 14:13:21 (7304): No heartbeat from core client for 30 sec - exiting 14:13:22 (7304): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8736, iMonCtr=1 Model crash detected, will try to restart... 15:27:52 (4556): No heartbeat from core client for 30 sec - exiting 15:27:53 (4556): No heartbeat from core client for 30 sec - exiting 15:27:54 (4556): No heartbeat from core client for 30 sec - exiting 15:27:55 (4556): No heartbeat from core client for 30 sec - exiting 15:27:56 (4556): No heartbeat from core client for 30 sec - exiting 15:27:57 (4556): No heartbeat from core client for 30 sec - exiting 15:27:58 (4556): No heartbeat from core client for 30 sec - exiting 15:27:59 (4556): No heartbeat from core client for 30 sec - exiting 15:28:01 (4556): No heartbeat from core client for 30 sec - exiting 15:28:02 (4556): No heartbeat from core client for 30 sec - exiting 15:28:03 (4556): No heartbeat from core client for 30 sec - exiting 15:28:04 (4556): No heartbeat from core client for 30 sec - exiting 15:28:05 (4556): No heartbeat from core client for 30 sec - exiting 15:28:06 (4556): No heartbeat from core client for 30 sec - exiting 15:28:07 (4556): No heartbeat from core client for 30 sec - exiting 15:28:08 (4556): No heartbeat from core client for 30 sec - exiting 15:28:09 (4556): No heartbeat from core client for 30 sec - exiting 15:28:10 (4556): No heartbeat from core client for 30 sec - exiting 15:28:11 (4556): No heartbeat from core client for 30 sec - exiting 15:28:12 (4556): No heartbeat from core client for 30 sec - exiting 15:28:13 (4556): No heartbeat from core client for 30 sec - exiting 15:28:14 (4556): No heartbeat from core client for 30 sec - exiting 15:28:15 (4556): No heartbeat from core client for 30 sec - exiting 15:28:16 (4556): No heartbeat from core client for 30 sec - exiting 15:28:17 (4556): No heartbeat from core client for 30 sec - exiting 15:28:18 (4556): No heartbeat from core client for 30 sec - exiting 15:28:19 (4556): No heartbeat from core client for 30 sec - exiting 15:28:20 (4556): No heartbeat from core client for 30 sec - exiting 15:28:21 (4556): No heartbeat from core client for 30 sec - exiting 15:28:22 (4556): No heartbeat from core client for 30 sec - exiting 15:28:23 (4556): No heartbeat from core client for 30 sec - exiting 15:28:24 (4556): No heartbeat from core client for 30 sec - exiting 15:28:25 (4556): No heartbeat from core client for 30 sec - exiting 15:28:26 (4556): No heartbeat from core client for 30 sec - exiting 15:28:27 (4556): No heartbeat from core client for 30 sec - exiting 15:28:28 (4556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:02:11 (4736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:02:12 (4736): No heartbeat from core client for 30 sec - exiting 07:02:13 (4736): No heartbeat from core client for 30 sec - exiting 07:02:14 (4736): No heartbeat from core client for 30 sec - exiting 07:02:15 (4736): No heartbeat from core client for 30 sec - exiting 07:02:16 (4736): No heartbeat from core client for 30 sec - exiting 07:02:17 (4736): No heartbeat from core client for 30 sec - exiting 07:02:18 (4736): No heartbeat from core client for 30 sec - exiting 07:02:19 (4736): No heartbeat from core client for 30 sec - exiting 07:02:20 (4736): No heartbeat from core client for 30 sec - exiting 07:05:19 (6916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:20 (6916): No heartbeat from core client for 30 sec - exiting 07:05:21 (6916): No heartbeat from core client for 30 sec - exiting 07:05:22 (6916): No heartbeat from core client for 30 sec - exiting 07:08:29 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:11:36 (6792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:11:37 (6792): No heartbeat from core client for 30 sec - exiting 07:11:38 (6792): No heartbeat from core client for 30 sec - exiting 07:11:39 (6792): No heartbeat from core client for 30 sec - exiting 07:11:40 (6792): No heartbeat from core client for 30 sec - exiting 07:11:41 (6792): No heartbeat from core client for 30 sec - exiting 07:11:42 (6792): No heartbeat from core client for 30 sec - exiting 07:20:49 (6340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:20:50 (6340): No heartbeat from core client for 30 sec - exiting 07:20:51 (6340): No heartbeat from core client for 30 sec - exiting 07:20:52 (6340): No heartbeat from core client for 30 sec - exiting 07:20:53 (6340): No heartbeat from core client for 30 sec - exiting 07:20:54 (6340): No heartbeat from core client for 30 sec - exiting 07:23:49 (3268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:36:01 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:54:12 (3688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:54:13 (3688): No heartbeat from core client for 30 sec - exiting 07:54:14 (3688): No heartbeat from core client for 30 sec - exiting 07:54:15 (3688): No heartbeat from core client for 30 sec - exiting 07:54:16 (3688): No heartbeat from core client for 30 sec - exiting 07:54:17 (3688): No heartbeat from core client for 30 sec - exiting 07:54:18 (3688): No heartbeat from core client for 30 sec - exiting 07:54:19 (3688): No heartbeat from core client for 30 sec - exiting 07:54:20 (3688): No heartbeat from core client for 30 sec - exiting 09:03:59 (6544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:00 (6544): No heartbeat from core client for 30 sec - exiting 09:04:01 (6544): No heartbeat from core client for 30 sec - exiting 09:04:02 (6544): No heartbeat from core client for 30 sec - exiting 09:04:03 (6544): No heartbeat from core client for 30 sec - exiting 09:13:16 (1220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:17 (1220): No heartbeat from core client for 30 sec - exiting 09:13:18 (1220): No heartbeat from core client for 30 sec - exiting 09:13:19 (1220): No heartbeat from core client for 30 sec - exiting 09:13:20 (1220): No heartbeat from core client for 30 sec - exiting 09:13:21 (1220): No heartbeat from core client for 30 sec - exiting 09:13:22 (1220): No heartbeat from core client for 30 sec - exiting 09:16:20 (3128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:16:21 (3128): No heartbeat from core client for 30 sec - exiting 09:16:22 (3128): No heartbeat from core client for 30 sec - exiting 09:16:23 (3128): No heartbeat from core client for 30 sec - exiting 09:16:24 (3128): No heartbeat from core client for 30 sec - exiting 09:16:25 (3128): No heartbeat from core client for 30 sec - exiting 09:16:26 (3128): No heartbeat from core client for 30 sec - exiting 09:16:27 (3128): No heartbeat from core client for 30 sec - exiting 09:22:22 (6924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:22:23 (6924): No heartbeat from core client for 30 sec - exiting 09:22:24 (6924): No heartbeat from core client for 30 sec - exiting 09:22:25 (6924): No heartbeat from core client for 30 sec - exiting 09:22:26 (6924): No heartbeat from core client for 30 sec - exiting 09:22:27 (6924): No heartbeat from core client for 30 sec - exiting 09:22:28 (6924): No heartbeat from core client for 30 sec - exiting 09:22:29 (6924): No heartbeat from core client for 30 sec - exiting 09:22:30 (6924): No heartbeat from core client for 30 sec - exiting 09:31:30 (6948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:31:31 (6948): No heartbeat from core client for 30 sec - exiting 09:34:30 (2708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:34:31 (2708): No heartbeat from core client for 30 sec - exiting 09:34:55 (6816): Can't acquire lockfile (32) - waiting 35s 09:37:35 (6816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:41 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:42 (5676): No heartbeat from core client for 30 sec - exiting 09:43:43 (5676): No heartbeat from core client for 30 sec - exiting 09:43:44 (5676): No heartbeat from core client for 30 sec - exiting 09:43:45 (5676): No heartbeat from core client for 30 sec - exiting 09:43:46 (5676): No heartbeat from core client for 30 sec - exiting 09:43:47 (5676): No heartbeat from core client for 30 sec - exiting 09:50:00 (6516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:50:01 (6516): No heartbeat from core client for 30 sec - exiting 09:50:02 (6516): No heartbeat from core client for 30 sec - exiting 10:38:56 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:57 (4564): No heartbeat from core client for 30 sec - exiting 10:38:58 (4564): No heartbeat from core client for 30 sec - exiting 10:38:59 (4564): No heartbeat from core client for 30 sec - exiting 10:39:00 (4564): No heartbeat from core client for 30 sec - exiting 10:39:01 (4564): No heartbeat from core client for 30 sec - exiting 10:39:02 (4564): No heartbeat from core client for 30 sec - exiting 10:42:01 (3720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:42:02 (3720): No heartbeat from core client for 30 sec - exiting 10:48:02 (6228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:16 (6340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:57:24 (2896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:57:25 (2896): No heartbeat from core client for 30 sec - exiting 10:57:26 (2896): No heartbeat from core client for 30 sec - exiting 10:57:27 (2896): No heartbeat from core client for 30 sec - exiting 10:57:28 (2896): No heartbeat from core client for 30 sec - exiting 10:57:29 (2896): No heartbeat from core client for 30 sec - exiting 10:57:30 (2896): No heartbeat from core client for 30 sec - exiting 10:57:31 (2896): No heartbeat from core client for 30 sec - exiting 11:00:28 (2360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:30 (5288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:31 (5288): No heartbeat from core client for 30 sec - exiting 11:03:32 (5288): No heartbeat from core client for 30 sec - exiting 11:03:33 (5288): No heartbeat from core client for 30 sec - exiting 11:03:34 (5288): No heartbeat from core client for 30 sec - exiting 11:03:35 (5288): No heartbeat from core client for 30 sec - exiting 11:03:36 (5288): No heartbeat from core client for 30 sec - exiting 11:03:37 (5288): No heartbeat from core client for 30 sec - exiting 11:03:38 (5288): No heartbeat from core client for 30 sec - exiting 11:03:39 (5288): No heartbeat from core client for 30 sec - exiting 11:03:40 (5288): No heartbeat from core client for 30 sec - exiting 11:15:42 (4760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:43 (4760): No heartbeat from core client for 30 sec - exiting 11:15:44 (4760): No heartbeat from core client for 30 sec - exiting 11:15:45 (4760): No heartbeat from core client for 30 sec - exiting 11:15:46 (4760): No heartbeat from core client for 30 sec - exiting 11:15:47 (4760): No heartbeat from core client for 30 sec - exiting 11:15:48 (4760): No heartbeat from core client for 30 sec - exiting 11:16:03 (2764): Can't acquire lockfile (32) - waiting 35s 11:18:44 (2764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:45 (2764): No heartbeat from core client for 30 sec - exiting 11:18:46 (2764): No heartbeat from core client for 30 sec - exiting 11:18:47 (2764): No heartbeat from core client for 30 sec - exiting 11:18:48 (2764): No heartbeat from core client for 30 sec - exiting 11:18:49 (2764): No heartbeat from core client for 30 sec - exiting 11:18:50 (2764): No heartbeat from core client for 30 sec - exiting 11:18:51 (2764): No heartbeat from core client for 30 sec - exiting 11:24:46 (6880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:24:47 (6880): No heartbeat from core client for 30 sec - exiting 11:24:48 (6880): No heartbeat from core client for 30 sec - exiting 11:24:49 (6880): No heartbeat from core client for 30 sec - exiting 11:24:50 (6880): No heartbeat from core client for 30 sec - exiting 11:24:51 (6880): No heartbeat from core client for 30 sec - exiting 11:24:52 (6880): No heartbeat from core client for 30 sec - exiting 11:25:05 (5996): Can't acquire lockfile (32) - waiting 35s Atmos Hold Restart file rename failed on atmos_restart.hold 11:30:52 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:30:53 (5996): No heartbeat from core client for 30 sec - exiting 11:30:54 (5996): No heartbeat from core client for 30 sec - exiting 11:30:55 (5996): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 11:37:00 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:37:01 (5652): No heartbeat from core client for 30 sec - exiting 11:37:02 (5652): No heartbeat from core client for 30 sec - exiting 11:37:03 (5652): No heartbeat from core client for 30 sec - exiting 11:37:04 (5652): No heartbeat from core client for 30 sec - exiting 11:37:05 (5652): No heartbeat from core client for 30 sec - exiting 11:37:06 (5652): No heartbeat from core client for 30 sec - exiting 11:37:07 (5652): No heartbeat from core client for 30 sec - exiting 11:37:08 (5652): No heartbeat from core client for 30 sec - exiting 11:43:07 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:43:08 (5104): No heartbeat from core client for 30 sec - exiting 11:43:09 (5104): No heartbeat from core client for 30 sec - exiting 11:43:10 (5104): No heartbeat from core client for 30 sec - exiting 11:43:11 (5104): No heartbeat from core client for 30 sec - exiting 11:43:12 (5104): No heartbeat from core client for 30 sec - exiting 11:43:13 (5104): No heartbeat from core client for 30 sec - exiting 11:43:14 (5104): No heartbeat from core client for 30 sec - exiting 11:43:15 (5104): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 11:52:15 (7152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:52:16 (7152): No heartbeat from core client for 30 sec - exiting 11:52:23 (7152): No heartbeat from core client for 30 sec - exiting 11:52:24 (7152): No heartbeat from core client for 30 sec - exiting 11:52:25 (7152): No heartbeat from core client for 30 sec - exiting 12:01:45 (6484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:48:28 (6696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6164, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:48:52 (3752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:48:53 (3752): No heartbeat from core client for 30 sec - exiting 23:48:54 (3752): No heartbeat from core client for 30 sec - exiting 06:45:55 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:22:43 (6308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:22:44 (6308): No heartbeat from core client for 30 sec - exiting 19:05:23 (1492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:38 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:39 (4612): No heartbeat from core client for 30 sec - exiting 19:26:40 (4612): No heartbeat from core client for 30 sec - exiting 19:26:41 (4612): No heartbeat from core client for 30 sec - exiting 19:26:42 (4612): No heartbeat from core client for 30 sec - exiting 19:26:43 (4612): No heartbeat from core client for 30 sec - exiting 19:26:44 (4612): No heartbeat from core client for 30 sec - exiting 19:32:41 (6280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:42 (6280): No heartbeat from core client for 30 sec - exiting 19:32:43 (6280): No heartbeat from core client for 30 sec - exiting 19:32:44 (6280): No heartbeat from core client for 30 sec - exiting 19:32:45 (6280): No heartbeat from core client for 30 sec - exiting 20:18:51 (6624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:12 (5172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:13 (5172): No heartbeat from core client for 30 sec - exiting 20:58:36 (6732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:15 (2016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:45 (4692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:46 (4692): No heartbeat from core client for 30 sec - exiting 21:23:47 (4692): No heartbeat from core client for 30 sec - exiting 21:26:49 (892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:27:02 (892): No heartbeat from core client for 30 sec - exiting 21:27:03 (892): No heartbeat from core client for 30 sec - exiting 21:27:04 (892): No heartbeat from core client for 30 sec - exiting 21:27:05 (892): No heartbeat from core client for 30 sec - exiting 21:27:06 (892): No heartbeat from core client for 30 sec - exiting 21:27:07 (892): No heartbeat from core client for 30 sec - exiting 21:27:08 (892): No heartbeat from core client for 30 sec - exiting 21:48:04 (3008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:48:05 (3008): No heartbeat from core client for 30 sec - exiting 21:48:06 (3008): No heartbeat from core client for 30 sec - exiting 21:57:10 (3464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:14 (6360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:15 (6360): No heartbeat from core client for 30 sec - exiting 22:00:16 (6360): No heartbeat from core client for 30 sec - exiting 22:03:18 (4764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:19 (4764): No heartbeat from core client for 30 sec - exiting 22:09:24 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:09:25 (3976): No heartbeat from core client for 30 sec - exiting 22:09:26 (3976): No heartbeat from core client for 30 sec - exiting 23:04:53 (6932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:54 (6932): No heartbeat from core client for 30 sec - exiting 23:04:55 (6932): No heartbeat from core client for 30 sec - exiting 23:04:56 (6932): No heartbeat from core client for 30 sec - exiting 00:11:32 (6328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:23:46 (6900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:32:51 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:38:55 (6884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:54:02 (6868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:09 (6372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:10 (6372): No heartbeat from core client for 30 sec - exiting 01:00:11 (6372): No heartbeat from core client for 30 sec - exiting 01:00:12 (6372): No heartbeat from core client for 30 sec - exiting 01:00:13 (6372): No heartbeat from core client for 30 sec - exiting 01:00:14 (6372): No heartbeat from core client for 30 sec - exiting 01:03:18 (2640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:03:19 (2640): No heartbeat from core client for 30 sec - exiting 01:03:20 (2640): No heartbeat from core client for 30 sec - exiting 01:03:21 (2640): No heartbeat from core client for 30 sec - exiting 01:03:22 (2640): No heartbeat from core client for 30 sec - exiting 01:03:23 (2640): No heartbeat from core client for 30 sec - exiting 01:06:25 (6332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:15:37 (7068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:15:38 (7068): No heartbeat from core client for 30 sec - exiting 01:15:39 (7068): No heartbeat from core client for 30 sec - exiting 01:15:40 (7068): No heartbeat from core client for 30 sec - exiting 01:18:43 (2796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:18:44 (2796): No heartbeat from core client for 30 sec - exiting 01:18:45 (2796): No heartbeat from core client for 30 sec - exiting 01:18:46 (2796): No heartbeat from core client for 30 sec - exiting 01:18:47 (2796): No heartbeat from core client for 30 sec - exiting 01:21:51 (1608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:21:52 (1608): No heartbeat from core client for 30 sec - exiting 01:21:53 (1608): No heartbeat from core client for 30 sec - exiting 01:21:54 (1608): No heartbeat from core client for 30 sec - exiting 01:21:55 (1608): No heartbeat from core client for 30 sec - exiting 01:21:56 (1608): No heartbeat from core client for 30 sec - exiting 01:21:57 (1608): No heartbeat from core client for 30 sec - exiting 01:34:03 (7132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:34:04 (7132): No heartbeat from core client for 30 sec - exiting 01:34:05 (7132): No heartbeat from core client for 30 sec - exiting 02:01:16 (6536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:01:22 (1180): Can't acquire lockfile (32) - waiting 35s 02:04:22 (1180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:04:39 (7036): Can't acquire lockfile (32) - waiting 35s 02:16:41 (7036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:41:05 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:41:06 (5604): No heartbeat from core client for 30 sec - exiting 02:53:09 (5504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:53:10 (5504): No heartbeat from core client for 30 sec - exiting 02:53:11 (5504): No heartbeat from core client for 30 sec - exiting 02:53:32 (6332): Can't acquire lockfile (32) - waiting 35s 02:56:13 (6332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:08:35 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:08:36 (5968): No heartbeat from core client for 30 sec - exiting 03:08:37 (5968): No heartbeat from core client for 30 sec - exiting 03:08:38 (5968): No heartbeat from core client for 30 sec - exiting 03:08:51 (6244): Can't acquire lockfile (32) - waiting 35s 03:11:36 (6244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:11:37 (6244): No heartbeat from core client for 30 sec - exiting 03:20:50 (6164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:23:57 (940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:23:58 (940): No heartbeat from core client for 30 sec - exiting 03:23:59 (940): No heartbeat from core client for 30 sec - exiting 03:24:00 (940): No heartbeat from core client for 30 sec - exiting 03:24:01 (940): No heartbeat from core client for 30 sec - exiting 03:24:02 (940): No heartbeat from core client for 30 sec - exiting 03:27:00 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:01 (6048): No heartbeat from core client for 30 sec - exiting 03:51:25 (6252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:57:27 (6696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:57:28 (6696): No heartbeat from core client for 30 sec - exiting 03:57:29 (6696): No heartbeat from core client for 30 sec - exiting 04:42:51 (5616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:42:52 (5616): No heartbeat from core client for 30 sec - exiting 06:48:03 (3056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:48:04 (3056): No heartbeat from core client for 30 sec - exiting 06:48:05 (3056): No heartbeat from core client for 30 sec - exiting 06:48:06 (3056): No heartbeat from core client for 30 sec - exiting 06:48:07 (3056): No heartbeat from core client for 30 sec - exiting 06:48:08 (3056): No heartbeat from core client for 30 sec - exiting 06:48:09 (3056): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:52:36 (2688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:52:37 (2688): No heartbeat from core client for 30 sec - exiting 22:52:38 (2688): No heartbeat from core client for 30 sec - exiting 22:52:39 (2688): No heartbeat from core client for 30 sec - exiting 22:52:40 (2688): No heartbeat from core client for 30 sec - exiting 22:52:41 (2688): No heartbeat from core client for 30 sec - exiting 22:52:42 (2688): No heartbeat from core client for 30 sec - exiting 22:52:43 (2688): No heartbeat from core client for 30 sec - exiting 22:52:44 (2688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Dec 2011 13:58:41	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	777,600	1,175,543	1.5118
27 Dec 2011 02:51:07	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	751,680	1,136,556	1.5120
26 Dec 2011 09:45:22	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	725,760	1,097,669	1.5124
25 Dec 2011 21:37:47	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	699,840	1,060,512	1.5154
25 Dec 2011 10:39:20	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	673,920	1,022,737	1.5176
24 Dec 2011 15:36:14	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	648,000	985,580	1.5210
24 Dec 2011 04:04:04	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	622,080	946,706	1.5218
23 Dec 2011 07:51:04	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	596,160	910,749	1.5277
22 Dec 2011 16:05:28	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	570,240	868,391	1.5229
22 Dec 2011 05:12:22	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	544,320	831,421	1.5274
21 Dec 2011 00:00:37	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	518,400	792,767	1.5293
13 Dec 2011 13:57:52	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	492,480	753,534	1.5301
12 Dec 2011 19:39:22	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	466,560	715,762	1.5341
10 Dec 2011 21:33:40	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	440,640	675,196	1.5323
05 Dec 2011 14:02:31	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	414,720	635,504	1.5324
01 Dec 2011 14:47:58	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	388,800	595,951	1.5328
28 Nov 2011 15:17:20	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	362,880	549,276	1.5137
27 Nov 2011 16:37:47	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	336,960	505,572	1.5004
27 Nov 2011 03:34:13	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	311,040	465,733	1.4973
26 Nov 2011 15:11:07	1123512	13638488	hadcm3n_o053_1980_40_007535804_2	285,120	426,025	1.4942