Task 14097405

Name	hadcm3n_u7as_1980_40_007544935_4
Workunit	7742167
Created	15 Feb 2012, 0:20:51 UTC
Sent	15 Feb 2012, 0:21:18 UTC
Report deadline	16 May 2012, 7:48:29 UTC
Received	12 Mar 2012, 14:16:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1122757
Run time	25 days 3 hours 29 min 35 sec
CPU time	24 days 6 hours 38 min 57 sec
Validate state	Invalid
Credit	4,354.56
Device peak FLOPS	1.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:27:18 (3324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 02:07:26 (5736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:32 (5736): No heartbeat from core client for 30 sec - exiting 02:07:33 (5736): No heartbeat from core client for 30 sec - exiting 02:07:35 (5736): No heartbeat from core client for 30 sec - exiting 02:13:27 (6040): No heartbeat from core client for 30 sec - exiting 02:13:36 (6040): No heartbeat from core client for 30 sec - exiting 02:13:37 (6040): No heartbeat from core client for 30 sec - exiting 02:13:38 (6040): No heartbeat from core client for 30 sec - exiting 02:13:39 (6040): No heartbeat from core client for 30 sec - exiting 02:13:40 (6040): No heartbeat from core client for 30 sec - exiting 02:13:41 (6040): No heartbeat from core client for 30 sec - exiting 02:13:42 (6040): No heartbeat from core client for 30 sec - exiting 02:13:44 (6040): No heartbeat from core client for 30 sec - exiting 02:13:45 (6040): No heartbeat from core client for 30 sec - exiting 02:13:46 (6040): No heartbeat from core client for 30 sec - exiting 02:13:47 (6040): No heartbeat from core client for 30 sec - exiting 02:13:48 (6040): No heartbeat from core client for 30 sec - exiting 02:13:49 (6040): No heartbeat from core client for 30 sec - exiting 02:13:50 (6040): No heartbeat from core client for 30 sec - exiting 02:13:51 (6040): No heartbeat from core client for 30 sec - exiting 02:13:52 (6040): No heartbeat from core client for 30 sec - exiting 02:13:53 (6040): No heartbeat from core client for 30 sec - exiting 02:13:54 (6040): No heartbeat from core client for 30 sec - exiting 02:13:56 (6040): No heartbeat from core client for 30 sec - exiting 02:13:57 (6040): No heartbeat from core client for 30 sec - exiting 02:13:58 (6040): No heartbeat from core client for 30 sec - exiting 02:13:59 (6040): No heartbeat from core client for 30 sec - exiting 02:14:00 (6040): No heartbeat from core client for 30 sec - exiting 02:14:01 (6040): No heartbeat from core client for 30 sec - exiting 02:14:02 (6040): No heartbeat from core client for 30 sec - exiting 02:14:03 (6040): No heartbeat from core client for 30 sec - exiting 02:14:04 (6040): No heartbeat from core client for 30 sec - exiting 02:14:05 (6040): No heartbeat from core client for 30 sec - exiting 02:14:06 (6040): No heartbeat from core client for 30 sec - exiting 02:14:08 (6040): No heartbeat from core client for 30 sec - exiting 02:14:09 (6040): No heartbeat from core client for 30 sec - exiting 02:14:10 (6040): No heartbeat from core client for 30 sec - exiting 02:14:11 (6040): No heartbeat from core client for 30 sec - exiting 02:14:12 (6040): No heartbeat from core client for 30 sec - exiting 02:14:13 (6040): No heartbeat from core client for 30 sec - exiting 02:14:14 (6040): No heartbeat from core client for 30 sec - exiting 02:14:15 (6040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:08 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:22 (6756): No heartbeat from core client for 30 sec - exiting 02:18:23 (6756): No heartbeat from core client for 30 sec - exiting 02:18:24 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 07:06:23 (4492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:10:41 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:10:47 (6048): No heartbeat from core client for 30 sec - exiting 02:44:32 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:44:38 (5796): No heartbeat from core client for 30 sec - exiting 02:44:39 (5796): No heartbeat from core client for 30 sec - exiting 02:44:40 (5796): No heartbeat from core client for 30 sec - exiting 02:44:41 (5796): No heartbeat from core client for 30 sec - exiting 02:44:42 (5796): No heartbeat from core client for 30 sec - exiting 02:44:43 (5796): No heartbeat from core client for 30 sec - exiting 02:44:44 (5796): No heartbeat from core client for 30 sec - exiting 02:44:45 (5796): No heartbeat from core client for 30 sec - exiting 02:44:46 (5796): No heartbeat from core client for 30 sec - exiting 02:44:47 (5796): No heartbeat from core client for 30 sec - exiting 02:44:49 (5796): No heartbeat from core client for 30 sec - exiting 02:44:50 (5796): No heartbeat from core client for 30 sec - exiting 02:44:51 (5796): No heartbeat from core client for 30 sec - exiting 02:44:52 (5796): No heartbeat from core client for 30 sec - exiting 02:44:53 (5796): No heartbeat from core client for 30 sec - exiting 02:44:54 (5796): No heartbeat from core client for 30 sec - exiting 02:44:55 (5796): No heartbeat from core client for 30 sec - exiting 02:44:56 (5796): No heartbeat from core client for 30 sec - exiting 02:44:57 (5796): No heartbeat from core client for 30 sec - exiting 02:44:58 (5796): No heartbeat from core client for 30 sec - exiting 02:45:00 (5796): No heartbeat from core client for 30 sec - exiting 02:45:01 (5796): No heartbeat from core client for 30 sec - exiting 02:45:02 (5796): No heartbeat from core client for 30 sec - exiting 02:45:03 (5796): No heartbeat from core client for 30 sec - exiting 02:45:04 (5796): No heartbeat from core client for 30 sec - exiting 02:45:09 (5796): No heartbeat from core client for 30 sec - exiting 02:45:10 (5796): No heartbeat from core client for 30 sec - exiting 02:45:11 (5796): No heartbeat from core client for 30 sec - exiting 02:45:12 (5796): No heartbeat from core client for 30 sec - exiting 02:45:13 (5796): No heartbeat from core client for 30 sec - exiting 02:45:14 (5796): No heartbeat from core client for 30 sec - exiting 02:45:15 (5796): No heartbeat from core client for 30 sec - exiting 02:45:16 (5796): No heartbeat from core client for 30 sec - exiting 18:31:35 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:31:37 (5036): No heartbeat from core client for 30 sec - exiting 18:31:38 (5036): No heartbeat from core client for 30 sec - exiting 18:31:39 (5036): No heartbeat from core client for 30 sec - exiting 01:49:10 (6556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:49:21 (6556): No heartbeat from core client for 30 sec - exiting 01:49:22 (6556): No heartbeat from core client for 30 sec - exiting 01:49:23 (6556): No heartbeat from core client for 30 sec - exiting 01:49:24 (6556): No heartbeat from core client for 30 sec - exiting 01:49:25 (6556): No heartbeat from core client for 30 sec - exiting 01:49:26 (6556): No heartbeat from core client for 30 sec - exiting 01:49:28 (6556): No heartbeat from core client for 30 sec - exiting 01:49:29 (6556): No heartbeat from core client for 30 sec - exiting 01:49:30 (6556): No heartbeat from core client for 30 sec - exiting 01:49:31 (6556): No heartbeat from core client for 30 sec - exiting 01:49:32 (6556): No heartbeat from core client for 30 sec - exiting 01:49:33 (6556): No heartbeat from core client for 30 sec - exiting 01:49:34 (6556): No heartbeat from core client for 30 sec - exiting 01:49:35 (6556): No heartbeat from core client for 30 sec - exiting 01:49:36 (6556): No heartbeat from core client for 30 sec - exiting 01:49:37 (6556): No heartbeat from core client for 30 sec - exiting 01:49:38 (6556): No heartbeat from core client for 30 sec - exiting 01:49:40 (6556): No heartbeat from core client for 30 sec - exiting 01:49:41 (6556): No heartbeat from core client for 30 sec - exiting 01:49:42 (6556): No heartbeat from core client for 30 sec - exiting 01:49:43 (6556): No heartbeat from core client for 30 sec - exiting 01:49:44 (6556): No heartbeat from core client for 30 sec - exiting 01:49:45 (6556): No heartbeat from core client for 30 sec - exiting 01:49:46 (6556): No heartbeat from core client for 30 sec - exiting 01:49:47 (6556): No heartbeat from core client for 30 sec - exiting 01:49:48 (6556): No heartbeat from core client for 30 sec - exiting 01:49:49 (6556): No heartbeat from core client for 30 sec - exiting 01:49:50 (6556): No heartbeat from core client for 30 sec - exiting 01:50:08 (6556): No heartbeat from core client for 30 sec - exiting 01:50:09 (6556): No heartbeat from core client for 30 sec - exiting 01:50:10 (6556): No heartbeat from core client for 30 sec - exiting 01:50:11 (6556): No heartbeat from core client for 30 sec - exiting 01:50:12 (6556): No heartbeat from core client for 30 sec - exiting 01:50:13 (6556): No heartbeat from core client for 30 sec - exiting 01:50:14 (6556): No heartbeat from core client for 30 sec - exiting 01:50:16 (6556): No heartbeat from core client for 30 sec - exiting 01:50:17 (6556): No heartbeat from core client for 30 sec - exiting 01:50:18 (6556): No heartbeat from core client for 30 sec - exiting 01:50:19 (6556): No heartbeat from core client for 30 sec - exiting 01:50:20 (6556): No heartbeat from core client for 30 sec - exiting 01:50:21 (6556): No heartbeat from core client for 30 sec - exiting 01:50:22 (6556): No heartbeat from core client for 30 sec - exiting 01:50:23 (6556): No heartbeat from core client for 30 sec - exiting 01:50:24 (6556): No heartbeat from core client for 30 sec - exiting 01:50:25 (6556): No heartbeat from core client for 30 sec - exiting 01:50:26 (6556): No heartbeat from core client for 30 sec - exiting 01:50:28 (6556): No heartbeat from core client for 30 sec - exiting 01:50:29 (6556): No heartbeat from core client for 30 sec - exiting 01:50:30 (6556): No heartbeat from core client for 30 sec - exiting 01:50:31 (6556): No heartbeat from core client for 30 sec - exiting 01:50:32 (6556): No heartbeat from core client for 30 sec - exiting 01:50:33 (6556): No heartbeat from core client for 30 sec - exiting 01:50:34 (6556): No heartbeat from core client for 30 sec - exiting 01:50:35 (6556): No heartbeat from core client for 30 sec - exiting 01:50:36 (6556): No heartbeat from core client for 30 sec - exiting 01:50:37 (6556): No heartbeat from core client for 30 sec - exiting 01:50:38 (6556): No heartbeat from core client for 30 sec - exiting 01:50:40 (6556): No heartbeat from core client for 30 sec - exiting 01:50:41 (6556): No heartbeat from core client for 30 sec - exiting 01:50:42 (6556): No heartbeat from core client for 30 sec - exiting 01:50:43 (6556): No heartbeat from core client for 30 sec - exiting 01:50:44 (6556): No heartbeat from core client for 30 sec - exiting 01:50:45 (6556): No heartbeat from core client for 30 sec - exiting 01:50:46 (6556): No heartbeat from core client for 30 sec - exiting 01:50:47 (6556): No heartbeat from core client for 30 sec - exiting 01:50:48 (6556): No heartbeat from core client for 30 sec - exiting 01:50:49 (6556): No heartbeat from core client for 30 sec - exiting 01:50:50 (6556): No heartbeat from core client for 30 sec - exiting 01:50:52 (6556): No heartbeat from core client for 30 sec - exiting 01:50:53 (6556): No heartbeat from core client for 30 sec - exiting 01:50:54 (6556): No heartbeat from core client for 30 sec - exiting 01:50:55 (6556): No heartbeat from core client for 30 sec - exiting 01:50:56 (6556): No heartbeat from core client for 30 sec - exiting 01:50:57 (6556): No heartbeat from core client for 30 sec - exiting 01:50:58 (6556): No heartbeat from core client for 30 sec - exiting 01:51:03 (6556): No heartbeat from core client for 30 sec - exiting 01:51:04 (6556): No heartbeat from core client for 30 sec - exiting 01:51:06 (6556): No heartbeat from core client for 30 sec - exiting 01:51:07 (6556): No heartbeat from core client for 30 sec - exiting 01:51:08 (6556): No heartbeat from core client for 30 sec - exiting 01:51:09 (6556): No heartbeat from core client for 30 sec - exiting 01:51:10 (6556): No heartbeat from core client for 30 sec - exiting 13:09:46 (3656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:09:54 (3656): No heartbeat from core client for 30 sec - exiting 13:09:55 (3656): No heartbeat from core client for 30 sec - exiting 13:09:56 (3656): No heartbeat from core client for 30 sec - exiting 13:09:57 (3656): No heartbeat from core client for 30 sec - exiting 13:09:58 (3656): No heartbeat from core client for 30 sec - exiting 13:09:59 (3656): No heartbeat from core client for 30 sec - exiting 13:10:00 (3656): No heartbeat from core client for 30 sec - exiting 13:10:01 (3656): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 20:57:36 (5408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:01:32 (5248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:55:15 (3036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:30 (3036): No heartbeat from core client for 30 sec - exiting 16:55:31 (3036): No heartbeat from core client for 30 sec - exiting 16:55:32 (3036): No heartbeat from core client for 30 sec - exiting 16:55:33 (3036): No heartbeat from core client for 30 sec - exiting 16:55:34 (3036): No heartbeat from core client for 30 sec - exiting 16:55:35 (3036): No heartbeat from core client for 30 sec - exiting 16:55:36 (3036): No heartbeat from core client for 30 sec - exiting 16:55:37 (3036): No heartbeat from core client for 30 sec - exiting 16:55:38 (3036): No heartbeat from core client for 30 sec - exiting 16:55:39 (3036): No heartbeat from core client for 30 sec - exiting 16:55:40 (3036): No heartbeat from core client for 30 sec - exiting 16:55:41 (3036): No heartbeat from core client for 30 sec - exiting 16:55:42 (3036): No heartbeat from core client for 30 sec - exiting 16:55:43 (3036): No heartbeat from core client for 30 sec - exiting 16:55:44 (3036): No heartbeat from core client for 30 sec - exiting 16:55:45 (3036): No heartbeat from core client for 30 sec - exiting 16:55:46 (3036): No heartbeat from core client for 30 sec - exiting 16:55:47 (3036): No heartbeat from core client for 30 sec - exiting 16:55:48 (3036): No heartbeat from core client for 30 sec - exiting 16:55:49 (3036): No heartbeat from core client for 30 sec - exiting 16:55:50 (3036): No heartbeat from core client for 30 sec - exiting 17:01:21 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:06:38 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:06:39 (5140): No heartbeat from core client for 30 sec - exiting 17:06:40 (5140): No heartbeat from core client for 30 sec - exiting 17:06:41 (5140): No heartbeat from core client for 30 sec - exiting 17:06:42 (5140): No heartbeat from core client for 30 sec - exiting 17:06:43 (5140): No heartbeat from core client for 30 sec - exiting 17:06:44 (5140): No heartbeat from core client for 30 sec - exiting 17:06:45 (5140): No heartbeat from core client for 30 sec - exiting 17:06:46 (5140): No heartbeat from core client for 30 sec - exiting 17:06:47 (5140): No heartbeat from core client for 30 sec - exiting 17:06:48 (5140): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 01:56:26 (3104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:13:22 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:35 (6816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:05:17 (5664): No heartbeat from core client for 30 sec - exiting 03:05:18 (5664): No heartbeat from core client for 30 sec - exiting 03:05:19 (5664): No heartbeat from core client for 30 sec - exiting 03:05:22 (5664): No heartbeat from core client for 30 sec - exiting 03:05:23 (5664): No heartbeat from core client for 30 sec - exiting 03:05:26 (5664): No heartbeat from core client for 30 sec - exiting 03:05:27 (5664): No heartbeat from core client for 30 sec - exiting 03:05:30 (5664): No heartbeat from core client for 30 sec - exiting 03:05:35 (5664): No heartbeat from core client for 30 sec - exiting 03:05:39 (5664): No heartbeat from core client for 30 sec - exiting 03:05:43 (5664): No heartbeat from core client for 30 sec - exiting 03:05:47 (5664): No heartbeat from core client for 30 sec - exiting 03:05:51 (5664): No heartbeat from core client for 30 sec - exiting 03:05:52 (5664): No heartbeat from core client for 30 sec - exiting 03:05:56 (5664): No heartbeat from core client for 30 sec - exiting 03:06:00 (5664): No heartbeat from core client for 30 sec - exiting 03:06:01 (5664): No heartbeat from core client for 30 sec - exiting 03:06:04 (5664): No heartbeat from core client for 30 sec - exiting 03:06:08 (5664): No heartbeat from core client for 30 sec - exiting 03:06:09 (5664): No heartbeat from core client for 30 sec - exiting 03:06:12 (5664): No heartbeat from core client for 30 sec - exiting 03:06:13 (5664): No heartbeat from core client for 30 sec - exiting 03:06:16 (5664): No heartbeat from core client for 30 sec - exiting 03:06:17 (5664): No heartbeat from core client for 30 sec - exiting 03:06:20 (5664): No heartbeat from core client for 30 sec - exiting 03:06:25 (5664): No heartbeat from core client for 30 sec - exiting 03:06:26 (5664): No heartbeat from core client for 30 sec - exiting 03:06:27 (5664): No heartbeat from core client for 30 sec - exiting 03:06:28 (5664): No heartbeat from core client for 30 sec - exiting 03:06:33 (5664): No heartbeat from core client for 30 sec - exiting 03:06:34 (5664): No heartbeat from core client for 30 sec - exiting 03:06:37 (5664): No heartbeat from core client for 30 sec - exiting 03:06:45 (5664): No heartbeat from core client for 30 sec - exiting 03:06:49 (5664): No heartbeat from core client for 30 sec - exiting 03:06:58 (5664): No heartbeat from core client for 30 sec - exiting 03:06:59 (5664): No heartbeat from core client for 30 sec - exiting 03:07:00 (5664): No heartbeat from core client for 30 sec - exiting 03:07:01 (5664): No heartbeat from core client for 30 sec - exiting 03:07:06 (5664): No heartbeat from core client for 30 sec - exiting 03:07:07 (5664): No heartbeat from core client for 30 sec - exiting 03:07:10 (5664): No heartbeat from core client for 30 sec - exiting 03:07:11 (5664): No heartbeat from core client for 30 sec - exiting 03:07:14 (5664): No heartbeat from core client for 30 sec - exiting 03:07:18 (5664): No heartbeat from core client for 30 sec - exiting 03:07:20 (5664): No heartbeat from core client for 30 sec - exiting 03:07:21 (5664): No heartbeat from core client for 30 sec - exiting 03:07:22 (5664): No heartbeat from core client for 30 sec - exiting 03:07:27 (5664): No heartbeat from core client for 30 sec - exiting 03:07:31 (5664): No heartbeat from core client for 30 sec - exiting 03:07:32 (5664): No heartbeat from core client for 30 sec - exiting 03:08:08 (5664): No heartbeat from core client for 30 sec - exiting 03:08:09 (5664): No heartbeat from core client for 30 sec - exiting 03:08:10 (5664): No heartbeat from core client for 30 sec - exiting 03:08:11 (5664): No heartbeat from core client for 30 sec - exiting 03:08:12 (5664): No heartbeat from core client for 30 sec - exiting 03:08:13 (5664): No heartbeat from core client for 30 sec - exiting 03:08:14 (5664): No heartbeat from core client for 30 sec - exiting 03:08:15 (5664): No heartbeat from core client for 30 sec - exiting 03:08:16 (5664): No heartbeat from core client for 30 sec - exiting 03:08:21 (5664): No heartbeat from core client for 30 sec - exiting 03:08:26 (5664): No heartbeat from core client for 30 sec - exiting 03:08:27 (5664): No heartbeat from core client for 30 sec - exiting 03:08:28 (5664): No heartbeat from core client for 30 sec - exiting 03:08:29 (5664): No heartbeat from core client for 30 sec - exiting 03:08:30 (5664): No heartbeat from core client for 30 sec - exiting 03:08:31 (5664): No heartbeat from core client for 30 sec - exiting 03:08:32 (5664): No heartbeat from core client for 30 sec - exiting 03:08:33 (5664): No heartbeat from core client for 30 sec - exiting 03:08:34 (5664): No heartbeat from core client for 30 sec - exiting 03:08:35 (5664): No heartbeat from core client for 30 sec - exiting 03:08:36 (5664): No heartbeat from core client for 30 sec - exiting 03:08:38 (5664): No heartbeat from core client for 30 sec - exiting 03:08:39 (5664): No heartbeat from core client for 30 sec - exiting 03:08:40 (5664): No heartbeat from core client for 30 sec - exiting 03:08:41 (5664): No heartbeat from core client for 30 sec - exiting 03:08:42 (5664): No heartbeat from core client for 30 sec - exiting 03:08:43 (5664): No heartbeat from core client for 30 sec - exiting 03:08:44 (5664): No heartbeat from core client for 30 sec - exiting 03:08:45 (5664): No heartbeat from core client for 30 sec - exiting 03:08:46 (5664): No heartbeat from core client for 30 sec - exiting 03:08:47 (5664): No heartbeat from core client for 30 sec - exiting 03:08:48 (5664): No heartbeat from core client for 30 sec - exiting 03:08:50 (5664): No heartbeat from core client for 30 sec - exiting 03:08:51 (5664): No heartbeat from core client for 30 sec - exiting 03:08:52 (5664): No heartbeat from core client for 30 sec - exiting 03:08:53 (5664): No heartbeat from core client for 30 sec - exiting 03:08:54 (5664): No heartbeat from core client for 30 sec - exiting 03:08:55 (5664): No heartbeat from core client for 30 sec - exiting 03:08:56 (5664): No heartbeat from core client for 30 sec - exiting 03:08:57 (5664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:11:21 (6000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:11:30 (6000): No heartbeat from core client for 30 sec - exiting 03:11:31 (6000): No heartbeat from core client for 30 sec - exiting 03:11:32 (6000): No heartbeat from core client for 30 sec - exiting 03:12:28 (4752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:33 (4752): No heartbeat from core client for 30 sec - exiting 03:12:34 (4752): No heartbeat from core client for 30 sec - exiting 03:12:35 (4752): No heartbeat from core client for 30 sec - exiting 03:13:35 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:45 (6220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:50 (6220): No heartbeat from core client for 30 sec - exiting 03:14:51 (6220): No heartbeat from core client for 30 sec - exiting 03:14:53 (6220): No heartbeat from core client for 30 sec - exiting 03:16:05 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:53:45 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:54:14 (4472): No heartbeat from core client for 30 sec - exiting 01:54:15 (4472): No heartbeat from core client for 30 sec - exiting 01:54:16 (4472): No heartbeat from core client for 30 sec - exiting 01:54:17 (4472): No heartbeat from core client for 30 sec - exiting 01:54:19 (4472): No heartbeat from core client for 30 sec - exiting 01:54:20 (4472): No heartbeat from core client for 30 sec - exiting 01:54:21 (4472): No heartbeat from core client for 30 sec - exiting 01:54:22 (4472): No heartbeat from core client for 30 sec - exiting 01:54:23 (4472): No heartbeat from core client for 30 sec - exiting 01:54:24 (4472): No heartbeat from core client for 30 sec - exiting 01:54:25 (4472): No heartbeat from core client for 30 sec - exiting 01:54:26 (4472): No heartbeat from core client for 30 sec - exiting 01:54:27 (4472): No heartbeat from core client for 30 sec - exiting 01:54:28 (4472): No heartbeat from core client for 30 sec - exiting 01:59:45 (3492): No heartbeat from core client for 30 sec - exiting 01:59:46 (3492): No heartbeat from core client for 30 sec - exiting 01:59:47 (3492): No heartbeat from core client for 30 sec - exiting 01:59:48 (3492): No heartbeat from core client for 30 sec - exiting 01:59:49 (3492): No heartbeat from core client for 30 sec - exiting 01:59:50 (3492): No heartbeat from core client for 30 sec - exiting 01:59:51 (3492): No heartbeat from core client for 30 sec - exiting 01:59:53 (3492): No heartbeat from core client for 30 sec - exiting 01:59:54 (3492): No heartbeat from core client for 30 sec - exiting 01:59:55 (3492): No heartbeat from core client for 30 sec - exiting 01:59:56 (3492): No heartbeat from core client for 30 sec - exiting 01:59:57 (3492): No heartbeat from core client for 30 sec - exiting 01:59:58 (3492): No heartbeat from core client for 30 sec - exiting 01:59:59 (3492): No heartbeat from core client for 30 sec - exiting 02:00:00 (3492): No heartbeat from core client for 30 sec - exiting 02:00:01 (3492): No heartbeat from core client for 30 sec - exiting 02:00:02 (3492): No heartbeat from core client for 30 sec - exiting 02:00:03 (3492): No heartbeat from core client for 30 sec - exiting 02:00:05 (3492): No heartbeat from core client for 30 sec - exiting 02:00:06 (3492): No heartbeat from core client for 30 sec - exiting 02:00:07 (3492): No heartbeat from core client for 30 sec - exiting 02:00:08 (3492): No heartbeat from core client for 30 sec - exiting 02:00:09 (3492): No heartbeat from core client for 30 sec - exiting 02:00:10 (3492): No heartbeat from core client for 30 sec - exiting 02:00:11 (3492): No heartbeat from core client for 30 sec - exiting 02:00:12 (3492): No heartbeat from core client for 30 sec - exiting 02:00:13 (3492): No heartbeat from core client for 30 sec - exiting 02:00:14 (3492): No heartbeat from core client for 30 sec - exiting 02:00:15 (3492): No heartbeat from core client for 30 sec - exiting 02:00:17 (3492): No heartbeat from core client for 30 sec - exiting 02:00:18 (3492): No heartbeat from core client for 30 sec - exiting 02:00:19 (3492): No heartbeat from core client for 30 sec - exiting 02:00:20 (3492): No heartbeat from core client for 30 sec - exiting 02:00:21 (3492): No heartbeat from core client for 30 sec - exiting 02:00:22 (3492): No heartbeat from core client for 30 sec - exiting 02:00:23 (3492): No heartbeat from core client for 30 sec - exiting 02:00:24 (3492): No heartbeat from core client for 30 sec - exiting 02:00:25 (3492): No heartbeat from core client for 30 sec - exiting 02:00:26 (3492): No heartbeat from core client for 30 sec - exiting 02:00:28 (3492): No heartbeat from core client for 30 sec - exiting 02:00:29 (3492): No heartbeat from core client for 30 sec - exiting 02:00:30 (3492): No heartbeat from core client for 30 sec - exiting 02:00:31 (3492): No heartbeat from core client for 30 sec - exiting 02:00:32 (3492): No heartbeat from core client for 30 sec - exiting 02:00:33 (3492): No heartbeat from core client for 30 sec - exiting 02:00:34 (3492): No heartbeat from core client for 30 sec - exiting 02:00:35 (3492): No heartbeat from core client for 30 sec - exiting 02:00:36 (3492): No heartbeat from core client for 30 sec - exiting 02:00:37 (3492): No heartbeat from core client for 30 sec - exiting 02:00:38 (3492): No heartbeat from core client for 30 sec - exiting 02:00:40 (3492): No heartbeat from core client for 30 sec - exiting 02:00:41 (3492): No heartbeat from core client for 30 sec - exiting 02:00:42 (3492): No heartbeat from core client for 30 sec - exiting 02:00:43 (3492): No heartbeat from core client for 30 sec - exiting 02:00:44 (3492): No heartbeat from core client for 30 sec - exiting 02:00:45 (3492): No heartbeat from core client for 30 sec - exiting 02:00:46 (3492): No heartbeat from core client for 30 sec - exiting 02:00:47 (3492): No heartbeat from core client for 30 sec - exiting 02:00:48 (3492): No heartbeat from core client for 30 sec - exiting 02:00:49 (3492): No heartbeat from core client for 30 sec - exiting 02:00:50 (3492): No heartbeat from core client for 30 sec - exiting 02:00:52 (3492): No heartbeat from core client for 30 sec - exiting 02:00:53 (3492): No heartbeat from core client for 30 sec - exiting 02:00:54 (3492): No heartbeat from core client for 30 sec - exiting 02:00:55 (3492): No heartbeat from core client for 30 sec - exiting 02:00:56 (3492): No heartbeat from core client for 30 sec - exiting 02:00:57 (3492): No heartbeat from core client for 30 sec - exiting 02:00:58 (3492): No heartbeat from core client for 30 sec - exiting 02:00:59 (3492): No heartbeat from core client for 30 sec - exiting 02:01:00 (3492): No heartbeat from core client for 30 sec - exiting 02:01:01 (3492): No heartbeat from core client for 30 sec - exiting 02:01:02 (3492): No heartbeat from core client for 30 sec - exiting 02:01:04 (3492): No heartbeat from core client for 30 sec - exiting 02:01:05 (3492): No heartbeat from core client for 30 sec - exiting 02:01:06 (3492): No heartbeat from core client for 30 sec - exiting 02:01:07 (3492): No heartbeat from core client for 30 sec - exiting 02:01:08 (3492): No heartbeat from core client for 30 sec - exiting 02:01:09 (3492): No heartbeat from core client for 30 sec - exiting 02:01:10 (3492): No heartbeat from core client for 30 sec - exiting 02:01:11 (3492): No heartbeat from core client for 30 sec - exiting 02:01:12 (3492): No heartbeat from core client for 30 sec - exiting 02:01:13 (3492): No heartbeat from core client for 30 sec - exiting 02:01:14 (3492): No heartbeat from core client for 30 sec - exiting 02:01:16 (3492): No heartbeat from core client for 30 sec - exiting 02:01:17 (3492): No heartbeat from core client for 30 sec - exiting 02:01:18 (3492): No heartbeat from core client for 30 sec - exiting 02:01:19 (3492): No heartbeat from core client for 30 sec - exiting 02:01:20 (3492): No heartbeat from core client for 30 sec - exiting 02:01:21 (3492): No heartbeat from core client for 30 sec - exiting 02:01:22 (3492): No heartbeat from core client for 30 sec - exiting 02:01:23 (3492): No heartbeat from core client for 30 sec - exiting 02:01:24 (3492): No heartbeat from core client for 30 sec - exiting 02:01:25 (3492): No heartbeat from core client for 30 sec - exiting 02:02:00 (3492): No heartbeat from core client for 30 sec - exiting 02:02:01 (3492): No heartbeat from core client for 30 sec - exiting 02:02:03 (3492): No heartbeat from core client for 30 sec - exiting 02:02:04 (3492): No heartbeat from core client for 30 sec - exiting 02:02:05 (3492): No heartbeat from core client for 30 sec - exiting 02:02:06 (3492): No heartbeat from core client for 30 sec - exiting 02:02:07 (3492): No heartbeat from core client for 30 sec - exiting 02:02:08 (3492): No heartbeat from core client for 30 sec - exiting 02:02:09 (3492): No heartbeat from core client for 30 sec - exiting 02:02:10 (3492): No heartbeat from core client for 30 sec - exiting 02:02:11 (3492): No heartbeat from core client for 30 sec - exiting 02:02:12 (3492): No heartbeat from core client for 30 sec - exiting 02:02:13 (3492): No heartbeat from core client for 30 sec - exiting 02:02:15 (3492): No heartbeat from core client for 30 sec - exiting 02:02:16 (3492): No heartbeat from core client for 30 sec - exiting 02:02:17 (3492): No heartbeat from core client for 30 sec - exiting 02:02:18 (3492): No heartbeat from core client for 30 sec - exiting 02:02:19 (3492): No heartbeat from core client for 30 sec - exiting 02:02:20 (3492): No heartbeat from core client for 30 sec - exiting 02:02:21 (3492): No heartbeat from core client for 30 sec - exiting 02:02:22 (3492): No heartbeat from core client for 30 sec - exiting 02:02:23 (3492): No heartbeat from core client for 30 sec - exiting 02:02:24 (3492): No heartbeat from core client for 30 sec - exiting 02:02:25 (3492): No heartbeat from core client for 30 sec - exiting 02:02:27 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:28 (3492): No heartbeat from core client for 30 sec - exiting 02:02:29 (3492): No heartbeat from core client for 30 sec - exiting 06:52:13 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Mar 2012 07:19:25	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	362,880	2,075,942	5.7207
11 Mar 2012 02:02:39	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	336,960	1,995,608	5.9224
10 Mar 2012 03:08:43	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	311,040	1,916,244	6.1608
08 Mar 2012 01:37:18	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	285,120	1,754,838	6.1547
06 Mar 2012 01:23:23	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	259,200	1,589,881	6.1338
04 Mar 2012 01:10:23	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	233,280	1,426,051	6.1130
02 Mar 2012 01:08:20	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	207,360	1,259,736	6.0751
29 Feb 2012 00:41:08	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	181,440	1,094,127	6.0302
27 Feb 2012 01:03:17	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	155,520	928,445	5.9699
25 Feb 2012 00:31:32	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	129,600	759,061	5.8570
23 Feb 2012 00:19:39	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	103,680	593,194	5.7214
21 Feb 2012 02:09:24	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	77,760	434,476	5.5874
19 Feb 2012 11:10:47	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	51,840	296,411	5.7178
17 Feb 2012 10:02:20	1122757	14097405	hadcm3n_u7as_1980_40_007544935_4	25,920	147,073	5.6741