Task 13548190

Name	hadcm3n_y9xm_1900_40_007521251_0
Workunit	7718726
Created	28 Oct 2011, 13:12:05 UTC
Sent	2 Nov 2011, 20:17:04 UTC
Report deadline	2 Feb 2012, 3:44:15 UTC
Received	9 Dec 2011, 17:02:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1183195
Run time	3 days 2 hours 4 min 4 sec
CPU time	2 days 11 hours 30 min 21 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	2.72 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:10:59 (5004): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:07:34 (5816): No heartbeat from core client for 30 sec - exiting 13:07:36 (5816): No heartbeat from core client for 30 sec - exiting 13:07:37 (5816): No heartbeat from core client for 30 sec - exiting 13:07:38 (5816): No heartbeat from core client for 30 sec - exiting 13:07:39 (5816): No heartbeat from core client for 30 sec - exiting 13:07:40 (5816): No heartbeat from core client for 30 sec - exiting 13:07:41 (5816): No heartbeat from core client for 30 sec - exiting 13:07:42 (5816): No heartbeat from core client for 30 sec - exiting 13:07:43 (5816): No heartbeat from core client for 30 sec - exiting 13:07:44 (5816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:16:12 (7588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:37:14 (3060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:14 (3084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:28:45 (7628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:55:53 (9136): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 18:55:54 (9136): No heartbeat from core client for 30 sec - exiting 18:55:55 (9136): No heartbeat from core client for 30 sec - exiting 18:55:56 (9136): No heartbeat from core client for 30 sec - exiting 18:55:57 (9136): No heartbeat from core client for 30 sec - exiting 18:55:58 (9136): No heartbeat from core client for 30 sec - exiting 18:55:59 (9136): No heartbeat from core client for 30 sec - exiting 18:56:00 (9136): No heartbeat from core client for 30 sec - exiting 18:56:01 (9136): No heartbeat from core client for 30 sec - exiting 18:56:02 (9136): No heartbeat from core client for 30 sec - exiting 18:56:03 (9136): No heartbeat from core client for 30 sec - exiting 18:56:04 (9136): No heartbeat from core client for 30 sec - exiting 18:56:05 (9136): No heartbeat from core client for 30 sec - exiting 18:56:06 (9136): No heartbeat from core client for 30 sec - exiting 21:13:40 (9712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:42 (9780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:41:37 (2364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:41:50 (2364): No heartbeat from core client for 30 sec - exiting 07:41:51 (2364): No heartbeat from core client for 30 sec - exiting 09:22:11 (4388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:22:12 (4388): No heartbeat from core client for 30 sec - exiting 09:35:52 (7260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:57 (7676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:54 (4288): No heartbeat from core client for 30 sec - exiting 09:52:55 (4288): No heartbeat from core client for 30 sec - exiting 09:52:56 (4288): No heartbeat from core client for 30 sec - exiting 09:52:57 (4288): No heartbeat from core client for 30 sec - exiting 09:52:58 (4288): No heartbeat from core client for 30 sec - exiting 09:52:59 (4288): No heartbeat from core client for 30 sec - exiting 09:53:00 (4288): No heartbeat from core client for 30 sec - exiting 09:53:01 (4288): No heartbeat from core client for 30 sec - exiting 09:53:02 (4288): No heartbeat from core client for 30 sec - exiting 09:53:03 (4288): No heartbeat from core client for 30 sec - exiting 09:53:04 (4288): No heartbeat from core client for 30 sec - exiting 09:53:05 (4288): No heartbeat from core client for 30 sec - exiting 09:53:06 (4288): No heartbeat from core client for 30 sec - exiting 09:53:07 (4288): No heartbeat from core client for 30 sec - exiting 09:53:08 (4288): No heartbeat from core client for 30 sec - exiting 09:53:09 (4288): No heartbeat from core client for 30 sec - exiting 09:53:10 (4288): No heartbeat from core client for 30 sec - exiting 09:53:11 (4288): No heartbeat from core client for 30 sec - exiting 09:53:12 (4288): No heartbeat from core client for 30 sec - exiting 09:53:13 (4288): No heartbeat from core client for 30 sec - exiting 09:53:14 (4288): No heartbeat from core client for 30 sec - exiting 09:53:15 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:01:07 (10404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:13 (11136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:14 (11136): No heartbeat from core client for 30 sec - exiting 14:49:15 (11136): No heartbeat from core client for 30 sec - exiting 17:00:00 (12024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:19 (9968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:38 (1172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:19 (10520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:15:10 (8228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:16:46 (3992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:33:18 (1948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:57 (10740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:58 (10740): No heartbeat from core client for 30 sec - exiting 22:14:59 (10740): No heartbeat from core client for 30 sec - exiting 22:15:00 (10740): No heartbeat from core client for 30 sec - exiting 22:15:01 (10740): No heartbeat from core client for 30 sec - exiting 22:15:02 (10740): No heartbeat from core client for 30 sec - exiting 22:15:03 (10740): No heartbeat from core client for 30 sec - exiting 22:15:04 (10740): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 22:20:08 (10404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:10 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:11 (5336): No heartbeat from core client for 30 sec - exiting 23:35:12 (5336): No heartbeat from core client for 30 sec - exiting 23:35:13 (5336): No heartbeat from core client for 30 sec - exiting 23:35:14 (5336): No heartbeat from core client for 30 sec - exiting 23:35:15 (5336): No heartbeat from core client for 30 sec - exiting 23:35:16 (5336): No heartbeat from core client for 30 sec - exiting 23:35:17 (5336): No heartbeat from core client for 30 sec - exiting 23:35:18 (5336): No heartbeat from core client for 30 sec - exiting 23:35:19 (5336): No heartbeat from core client for 30 sec - exiting 23:35:20 (5336): No heartbeat from core client for 30 sec - exiting 23:42:14 (9392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:47:28 (7840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:49:27 (7768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:11:55 (8240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:55:28 (7808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:39 (12396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:40 (12396): No heartbeat from core client for 30 sec - exiting 01:23:41 (12396): No heartbeat from core client for 30 sec - exiting 01:23:42 (12396): No heartbeat from core client for 30 sec - exiting 01:26:09 (11992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:08:25 (11476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:19:55 (11024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:23:06 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:28:40 (12332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:46:19 (5680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:47:21 (9024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:58:43 (8156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:15:10 (10584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:39:27 (10616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:40:43 (7640): No heartbeat from core client for 30 sec - exiting 03:40:44 (7640): No heartbeat from core client for 30 sec - exiting 03:40:45 (7640): No heartbeat from core client for 30 sec - exiting 03:40:46 (7640): No heartbeat from core client for 30 sec - exiting 03:40:47 (7640): No heartbeat from core client for 30 sec - exiting 03:40:48 (7640): No heartbeat from core client for 30 sec - exiting 03:40:49 (7640): No heartbeat from core client for 30 sec - exiting 03:40:50 (7640): No heartbeat from core client for 30 sec - exiting 03:40:51 (7640): No heartbeat from core client for 30 sec - exiting 03:40:52 (7640): No heartbeat from core client for 30 sec - exiting 03:40:53 (7640): No heartbeat from core client for 30 sec - exiting 03:40:54 (7640): No heartbeat from core client for 30 sec - exiting 03:40:55 (7640): No heartbeat from core client for 30 sec - exiting 03:40:56 (7640): No heartbeat from core client for 30 sec - exiting 03:40:57 (7640): No heartbeat from core client for 30 sec - exiting 03:40:58 (7640): No heartbeat from core client for 30 sec - exiting 03:40:59 (7640): No heartbeat from core client for 30 sec - exiting 03:41:00 (7640): No heartbeat from core client for 30 sec - exiting 03:41:01 (7640): No heartbeat from core client for 30 sec - exiting 03:41:02 (7640): No heartbeat from core client for 30 sec - exiting 03:41:03 (7640): No heartbeat from core client for 30 sec - exiting 03:41:04 (7640): No heartbeat from core client for 30 sec - exiting 03:41:05 (7640): No heartbeat from core client for 30 sec - exiting 03:41:06 (7640): No heartbeat from core client for 30 sec - exiting 03:41:07 (7640): No heartbeat from core client for 30 sec - exiting 03:41:08 (7640): No heartbeat from core client for 30 sec - exiting 03:41:09 (7640): No heartbeat from core client for 30 sec - exiting 03:41:10 (7640): No heartbeat from core client for 30 sec - exiting 03:41:11 (7640): No heartbeat from core client for 30 sec - exiting 03:41:12 (7640): No heartbeat from core client for 30 sec - exiting 03:41:13 (7640): No heartbeat from core client for 30 sec - exiting 03:41:14 (7640): No heartbeat from core client for 30 sec - exiting 03:41:15 (7640): No heartbeat from core client for 30 sec - exiting 03:41:16 (7640): No heartbeat from core client for 30 sec - exiting 03:41:17 (7640): No heartbeat from core client for 30 sec - exiting 03:41:18 (7640): No heartbeat from core client for 30 sec - exiting 03:41:19 (7640): No heartbeat from core client for 30 sec - exiting 03:41:20 (7640): No heartbeat from core client for 30 sec - exiting 03:41:21 (7640): No heartbeat from core client for 30 sec - exiting 03:41:22 (7640): No heartbeat from core client for 30 sec - exiting 03:41:23 (7640): No heartbeat from core client for 30 sec - exiting 03:41:24 (7640): No heartbeat from core client for 30 sec - exiting 03:41:25 (7640): No heartbeat from core client for 30 sec - exiting 03:41:26 (7640): No heartbeat from core client for 30 sec - exiting 03:41:27 (7640): No heartbeat from core client for 30 sec - exiting 03:41:28 (7640): No heartbeat from core client for 30 sec - exiting 03:41:29 (7640): No heartbeat from core client for 30 sec - exiting 03:41:30 (7640): No heartbeat from core client for 30 sec - exiting 03:41:31 (7640): No heartbeat from core client for 30 sec - exiting 03:41:32 (7640): No heartbeat from core client for 30 sec - exiting 03:41:33 (7640): No heartbeat from core client for 30 sec - exiting 03:41:34 (7640): No heartbeat from core client for 30 sec - exiting 03:41:35 (7640): No heartbeat from core client for 30 sec - exiting 03:41:36 (7640): No heartbeat from core client for 30 sec - exiting 03:41:37 (7640): No heartbeat from core client for 30 sec - exiting 03:41:38 (7640): No heartbeat from core client for 30 sec - exiting 03:41:39 (7640): No heartbeat from core client for 30 sec - exiting 03:41:41 (7640): No heartbeat from core client for 30 sec - exiting 03:41:42 (7640): No heartbeat from core client for 30 sec - exiting 03:41:43 (7640): No heartbeat from core client for 30 sec - exiting 03:41:44 (7640): No heartbeat from core client for 30 sec - exiting 03:41:45 (7640): No heartbeat from core client for 30 sec - exiting 03:41:46 (7640): No heartbeat from core client for 30 sec - exiting 03:41:47 (7640): No heartbeat from core client for 30 sec - exiting 03:41:50 (7640): No heartbeat from core client for 30 sec - exiting 03:41:51 (7640): No heartbeat from core client for 30 sec - exiting 03:41:52 (7640): No heartbeat from core client for 30 sec - exiting 03:41:53 (7640): No heartbeat from core client for 30 sec - exiting 03:41:54 (7640): No heartbeat from core client for 30 sec - exiting 03:41:55 (7640): No heartbeat from core client for 30 sec - exiting 03:41:56 (7640): No heartbeat from core client for 30 sec - exiting 03:41:57 (7640): No heartbeat from core client for 30 sec - exiting 03:41:58 (7640): No heartbeat from core client for 30 sec - exiting 03:41:59 (7640): No heartbeat from core client for 30 sec - exiting 03:42:00 (7640): No heartbeat from core client for 30 sec - exiting 03:42:01 (7640): No heartbeat from core client for 30 sec - exiting 03:42:02 (7640): No heartbeat from core client for 30 sec - exiting 03:42:04 (7640): No heartbeat from core client for 30 sec - exiting 03:42:05 (7640): No heartbeat from core client for 30 sec - exiting 03:42:07 (7640): No heartbeat from core client for 30 sec - exiting 03:42:08 (7640): No heartbeat from core client for 30 sec - exiting 03:42:09 (7640): No heartbeat from core client for 30 sec - exiting 03:42:10 (7640): No heartbeat from core client for 30 sec - exiting 03:42:13 (7640): No heartbeat from core client for 30 sec - exiting 03:42:14 (7640): No heartbeat from core client for 30 sec - exiting 03:42:15 (7640): No heartbeat from core client for 30 sec - exiting 03:42:16 (7640): No heartbeat from core client for 30 sec - exiting 03:42:17 (7640): No heartbeat from core client for 30 sec - exiting 03:42:19 (7640): No heartbeat from core client for 30 sec - exiting 03:42:20 (7640): No heartbeat from core client for 30 sec - exiting 03:42:21 (7640): No heartbeat from core client for 30 sec - exiting 03:42:22 (7640): No heartbeat from core client for 30 sec - exiting 03:42:23 (7640): No heartbeat from core client for 30 sec - exiting 03:42:24 (7640): No heartbeat from core client for 30 sec - exiting 03:42:25 (7640): No heartbeat from core client for 30 sec - exiting 03:42:26 (7640): No heartbeat from core client for 30 sec - exiting 03:42:27 (7640): No heartbeat from core client for 30 sec - exiting 03:42:28 (7640): No heartbeat from core client for 30 sec - exiting 03:42:29 (7640): No heartbeat from core client for 30 sec - exiting 03:42:32 (7640): No heartbeat from core client for 30 sec - exiting 03:42:33 (7640): No heartbeat from core client for 30 sec - exiting 03:42:34 (7640): No heartbeat from core client for 30 sec - exiting 03:42:35 (7640): No heartbeat from core client for 30 sec - exiting 03:42:36 (7640): No heartbeat from core client for 30 sec - exiting 03:42:37 (7640): No heartbeat from core client for 30 sec - exiting 03:42:38 (7640): No heartbeat from core client for 30 sec - exiting 03:42:39 (7640): No heartbeat from core client for 30 sec - exiting 03:42:40 (7640): No heartbeat from core client for 30 sec - exiting 03:42:41 (7640): No heartbeat from core client for 30 sec - exiting 03:42:42 (7640): No heartbeat from core client for 30 sec - exiting 03:42:43 (7640): No heartbeat from core client for 30 sec - exiting 03:42:44 (7640): No heartbeat from core client for 30 sec - exiting 03:42:45 (7640): No heartbeat from core client for 30 sec - exiting 03:42:46 (7640): No heartbeat from core client for 30 sec - exiting 03:42:47 (7640): No heartbeat from core client for 30 sec - exiting 03:42:48 (7640): No heartbeat from core client for 30 sec - exiting 03:42:49 (7640): No heartbeat from core client for 30 sec - exiting 03:42:50 (7640): No heartbeat from core client for 30 sec - exiting 03:42:51 (7640): No heartbeat from core client for 30 sec - exiting 03:42:52 (7640): No heartbeat from core client for 30 sec - exiting 03:42:54 (7640): No heartbeat from core client for 30 sec - exiting 03:42:55 (7640): No heartbeat from core client for 30 sec - exiting 03:42:56 (7640): No heartbeat from core client for 30 sec - exiting 03:42:57 (7640): No heartbeat from core client for 30 sec - exiting 03:42:58 (7640): No heartbeat from core client for 30 sec - exiting 03:43:00 (7640): No heartbeat from core client for 30 sec - exiting 03:43:01 (7640): No heartbeat from core client for 30 sec - exiting 03:43:03 (7640): No heartbeat from core client for 30 sec - exiting 03:43:04 (7640): No heartbeat from core client for 30 sec - exiting 03:43:05 (7640): No heartbeat from core client for 30 sec - exiting 03:43:06 (7640): No heartbeat from core client for 30 sec - exiting 03:43:07 (7640): No heartbeat from core client for 30 sec - exiting 03:43:08 (7640): No heartbeat from core client for 30 sec - exiting 03:43:09 (7640): No heartbeat from core client for 30 sec - exiting 03:43:10 (7640): No heartbeat from core client for 30 sec - exiting 03:43:11 (7640): No heartbeat from core client for 30 sec - exiting 03:43:12 (7640): No heartbeat from core client for 30 sec - exiting 03:43:13 (7640): No heartbeat from core client for 30 sec - exiting 03:43:15 (7640): No heartbeat from core client for 30 sec - exiting 03:43:16 (7640): No heartbeat from core client for 30 sec - exiting 03:43:17 (7640): No heartbeat from core client for 30 sec - exiting 03:43:18 (7640): No heartbeat from core client for 30 sec - exiting 03:43:19 (7640): No heartbeat from core client for 30 sec - exiting 03:43:20 (7640): No heartbeat from core client for 30 sec - exiting 03:43:21 (7640): No heartbeat from core client for 30 sec - exiting 03:43:22 (7640): No heartbeat from core client for 30 sec - exiting 03:43:24 (7640): No heartbeat from core client for 30 sec - exiting 03:43:25 (7640): No heartbeat from core client for 30 sec - exiting 03:43:26 (7640): No heartbeat from core client for 30 sec - exiting 03:43:27 (7640): No heartbeat from core client for 30 sec - exiting 03:43:29 (7640): No heartbeat from core client for 30 sec - exiting 03:43:30 (7640): No heartbeat from core client for 30 sec - exiting 03:43:31 (7640): No heartbeat from core client for 30 sec - exiting 03:43:32 (7640): No heartbeat from core client for 30 sec - exiting 03:43:33 (7640): No heartbeat from core client for 30 sec - exiting 03:43:35 (7640): No heartbeat from core client for 30 sec - exiting 03:43:36 (7640): No heartbeat from core client for 30 sec - exiting 03:43:37 (7640): No heartbeat from core client for 30 sec - exiting 03:43:38 (7640): No heartbeat from core client for 30 sec - exiting 03:43:39 (7640): No heartbeat from core client for 30 sec - exiting 03:43:40 (7640): No heartbeat from core client for 30 sec - exiting 03:43:41 (7640): No heartbeat from core client for 30 sec - exiting 03:43:42 (7640): No heartbeat from core client for 30 sec - exiting 03:43:43 (7640): No heartbeat from core client for 30 sec - exiting 03:43:44 (7640): No heartbeat from core client for 30 sec - exiting 03:43:45 (7640): No heartbeat from core client for 30 sec - exiting 03:43:46 (7640): No heartbeat from core client for 30 sec - exiting 03:43:47 (7640): No heartbeat from core client for 30 sec - exiting 03:43:48 (7640): No heartbeat from core client for 30 sec - exiting 03:43:49 (7640): No heartbeat from core client for 30 sec - exiting 03:43:50 (7640): No heartbeat from core client for 30 sec - exiting 03:43:52 (7640): No heartbeat from core client for 30 sec - exiting 03:43:53 (7640): No heartbeat from core client for 30 sec - exiting 03:43:54 (7640): No heartbeat from core client for 30 sec - exiting 03:43:55 (7640): No heartbeat from core client for 30 sec - exiting 03:43:56 (7640): No heartbeat from core client for 30 sec - exiting 03:43:58 (7640): No heartbeat from core client for 30 sec - exiting 03:43:59 (7640): No heartbeat from core client for 30 sec - exiting 03:44:00 (7640): No heartbeat from core client for 30 sec - exiting 03:44:01 (7640): No heartbeat from core client for 30 sec - exiting 03:44:02 (7640): No heartbeat from core client for 30 sec - exiting 03:44:04 (7640): No heartbeat from core client for 30 sec - exiting 03:44:05 (7640): No heartbeat from core client for 30 sec - exiting 03:44:06 (7640): No heartbeat from core client for 30 sec - exiting 03:44:07 (7640): No heartbeat from core client for 30 sec - exiting 03:44:08 (7640): No heartbeat from core client for 30 sec - exiting 03:44:09 (7640): No heartbeat from core client for 30 sec - exiting 03:44:10 (7640): No heartbeat from core client for 30 sec - exiting 03:44:11 (7640): No heartbeat from core client for 30 sec - exiting 03:44:12 (7640): No heartbeat from core client for 30 sec - exiting 03:44:13 (7640): No heartbeat from core client for 30 sec - exiting 03:44:14 (7640): No heartbeat from core client for 30 sec - exiting 03:44:15 (7640): No heartbeat from core client for 30 sec - exiting 03:44:16 (7640): No heartbeat from core client for 30 sec - exiting 03:44:17 (7640): No heartbeat from core client for 30 sec - exiting 03:44:20 (7640): No heartbeat from core client for 30 sec - exiting 03:44:21 (7640): No heartbeat from core client for 30 sec - exiting 03:44:22 (7640): No heartbeat from core client for 30 sec - exiting 03:44:23 (7640): No heartbeat from core client for 30 sec - exiting 03:44:24 (7640): No heartbeat from core client for 30 sec - exiting 03:44:25 (7640): No heartbeat from core client for 30 sec - exiting 03:44:33 (7640): No heartbeat from core client for 30 sec - exiting 03:44:34 (7640): No heartbeat from core client for 30 sec - exiting 03:44:35 (7640): No heartbeat from core client for 30 sec - exiting 03:44:36 (7640): No heartbeat from core client for 30 sec - exiting 03:44:37 (7640): No heartbeat from core client for 30 sec - exiting 03:44:38 (7640): No heartbeat from core client for 30 sec - exiting 03:44:39 (7640): No heartbeat from core client for 30 sec - exiting 03:44:40 (7640): No heartbeat from core client for 30 sec - exiting 03:44:41 (7640): No heartbeat from core client for 30 sec - exiting 03:44:43 (7640): No heartbeat from core client for 30 sec - exiting 03:44:44 (7640): No heartbeat from core client for 30 sec - exiting 03:44:45 (7640): No heartbeat from core client for 30 sec - exiting 03:44:46 (7640): No heartbeat from core client for 30 sec - exiting 03:44:47 (7640): No heartbeat from core client for 30 sec - exiting 03:44:48 (7640): No heartbeat from core client for 30 sec - exiting 03:44:49 (7640): No heartbeat from core client for 30 sec - exiting 03:44:50 (7640): No heartbeat from core client for 30 sec - exiting 03:44:51 (7640): No heartbeat from core client for 30 sec - exiting 03:44:52 (7640): No heartbeat from core client for 30 sec - exiting 03:44:53 (7640): No heartbeat from core client for 30 sec - exiting 03:44:54 (7640): No heartbeat from core client for 30 sec - exiting 03:44:55 (7640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:30 (8188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:36:35 (10336): No heartbeat from core client for 30 sec - exiting 05:36:36 (10336): No heartbeat from core client for 30 sec - exiting 05:36:37 (10336): No heartbeat from core client for 30 sec - exiting 05:36:38 (10336): No heartbeat from core client for 30 sec - exiting 05:36:39 (10336): No heartbeat from core client for 30 sec - exiting 05:36:40 (10336): No heartbeat from core client for 30 sec - exiting 05:36:41 (10336): No heartbeat from core client for 30 sec - exiting 05:36:42 (10336): No heartbeat from core client for 30 sec - exiting 05:36:43 (10336): No heartbeat from core client for 30 sec - exiting 05:36:44 (10336): No heartbeat from core client for 30 sec - exiting 05:36:45 (10336): No heartbeat from core client for 30 sec - exiting 05:36:46 (10336): No heartbeat from core client for 30 sec - exiting 05:36:47 (10336): No heartbeat from core client for 30 sec - exiting 05:36:48 (10336): No heartbeat from core client for 30 sec - exiting 05:36:49 (10336): No heartbeat from core client for 30 sec - exiting 05:36:50 (10336): No heartbeat from core client for 30 sec - exiting 05:36:51 (10336): No heartbeat from core client for 30 sec - exiting 05:36:52 (10336): No heartbeat from core client for 30 sec - exiting 05:36:53 (10336): No heartbeat from core client for 30 sec - exiting 05:36:54 (10336): No heartbeat from core client for 30 sec - exiting 05:36:55 (10336): No heartbeat from core client for 30 sec - exiting 05:36:56 (10336): No heartbeat from core client for 30 sec - exiting 05:37:00 (10336): No heartbeat from core client for 30 sec - exiting 05:37:01 (10336): No heartbeat from core client for 30 sec - exiting 05:38:01 (10336): No heartbeat from core client for 30 sec - exiting 05:38:02 (10336): No heartbeat from core client for 30 sec - exiting 05:38:03 (10336): No heartbeat from core client for 30 sec - exiting 05:38:04 (10336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:47 (10248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:12:43 (8348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:20:07 (11872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:33:55 (8300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:32:18 (5580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:39:12 (12572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:42:51 (10488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2064, iMonCtr=1 Model crash detected, will try to restart... 11:35:39 (2064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6344, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6344, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6344, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6344, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6344, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Dec 2011 14:17:50	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	414,720	757,946	1.8276
01 Dec 2011 04:24:16	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	388,800	723,572	1.8610
30 Nov 2011 18:19:34	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	362,880	689,035	1.8988
30 Nov 2011 08:41:35	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	336,960	655,403	1.9450
29 Nov 2011 17:03:27	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	311,040	606,607	1.9503
29 Nov 2011 02:24:00	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	285,120	557,156	1.9541
28 Nov 2011 10:00:58	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	259,200	506,632	1.9546
25 Nov 2011 16:37:56	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	233,280	457,240	1.9600
23 Nov 2011 22:28:34	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	207,360	402,776	1.9424
17 Nov 2011 10:43:53	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	181,440	349,742	1.9276
15 Nov 2011 17:32:06	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	155,520	299,053	1.9229
15 Nov 2011 17:32:06	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	129,600	248,847	1.9201
10 Nov 2011 07:14:58	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	103,680	199,677	1.9259
08 Nov 2011 01:17:37	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	77,760	150,106	1.9304
06 Nov 2011 08:09:00	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	51,840	100,136	1.9316
04 Nov 2011 13:53:12	1177897	13548190	hadcm3n_y9xm_1900_40_007521251_0	25,920	50,141	1.9345