Task 15426137

Name	hadcm3n_o4hm_2060_40_008239563_3
Workunit	8394687
Created	3 Nov 2012, 14:06:04 UTC
Sent	3 Nov 2012, 14:06:09 UTC
Report deadline	2 Feb 2013, 21:33:20 UTC
Received	27 Nov 2012, 5:11:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	912411
Run time	11 days 17 hours 7 min 16 sec
CPU time	11 days 17 hours 7 min 16 sec
Validate state	Invalid
Credit	5,909.76
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.2.19</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 00:01:56 (3364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:57 (3364): No heartbeat from core client for 30 sec - exiting 00:01:58 (3364): No heartbeat from core client for 30 sec - exiting 00:03:32 (7048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:03:33 (7048): No heartbeat from core client for 30 sec - exiting 00:03:34 (7048): No heartbeat from core client for 30 sec - exiting 00:03:35 (7048): No heartbeat from core client for 30 sec - exiting 00:03:36 (7048): No heartbeat from core client for 30 sec - exiting 00:03:38 (7048): No heartbeat from core client for 30 sec - exiting 00:03:39 (7048): No heartbeat from core client for 30 sec - exiting 00:03:40 (7048): No heartbeat from core client for 30 sec - exiting 00:03:41 (7048): No heartbeat from core client for 30 sec - exiting 00:03:42 (7048): No heartbeat from core client for 30 sec - exiting 00:03:43 (7048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:45:57 (7400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:45:58 (7400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:35:08 (11896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:16:39 (8604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:04:54 (8840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:56 (9720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:58 (9720): No heartbeat from core client for 30 sec - exiting 04:14:59 (9720): No heartbeat from core client for 30 sec - exiting 04:16:39 (7560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:18:34 (9912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:18:35 (9912): No heartbeat from core client for 30 sec - exiting 16:20:26 (12964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:02:12 (11964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:04 (12188): No heartbeat from core client for 30 sec - exiting 22:03:05 (12188): No heartbeat from core client for 30 sec - exiting 22:03:07 (12188): No heartbeat from core client for 30 sec - exiting 22:03:08 (12188): No heartbeat from core client for 30 sec - exiting 22:03:09 (12188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:17:40 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:17:42 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:22:04 (6936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:17:06 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:17:07 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 01:37:37 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:37:38 (4456): No heartbeat from core client for 30 sec - exiting 01:38:54 (7364): No heartbeat from core client for 30 sec - exiting 01:38:55 (7364): No heartbeat from core client for 30 sec - exiting 01:38:56 (7364): No heartbeat from core client for 30 sec - exiting 01:38:57 (7364): No heartbeat from core client for 30 sec - exiting 01:38:58 (7364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:38:59 (7364): No heartbeat from core client for 30 sec - exiting 01:39:00 (7364): No heartbeat from core client for 30 sec - exiting 01:39:01 (7364): No heartbeat from core client for 30 sec - exiting 01:39:03 (7364): No heartbeat from core client for 30 sec - exiting 01:39:04 (7364): No heartbeat from core client for 30 sec - exiting 01:39:05 (7364): No heartbeat from core client for 30 sec - exiting 01:39:06 (7364): No heartbeat from core client for 30 sec - exiting 01:39:07 (7364): No heartbeat from core client for 30 sec - exiting 01:39:08 (7364): No heartbeat from core client for 30 sec - exiting 01:39:09 (7364): No heartbeat from core client for 30 sec - exiting 01:47:37 (2408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:47:39 (2408): No heartbeat from core client for 30 sec - exiting 01:47:40 (2408): No heartbeat from core client for 30 sec - exiting 01:49:17 (5796): No heartbeat from core client for 30 sec - exiting 01:49:18 (5796): No heartbeat from core client for 30 sec - exiting 01:49:19 (5796): No heartbeat from core client for 30 sec - exiting 01:49:21 (5796): No heartbeat from core client for 30 sec - exiting 01:49:22 (5796): No heartbeat from core client for 30 sec - exiting 01:49:23 (5796): No heartbeat from core client for 30 sec - exiting 01:49:24 (5796): No heartbeat from core client for 30 sec - exiting 01:49:25 (5796): No heartbeat from core client for 30 sec - exiting 01:49:26 (5796): No heartbeat from core client for 30 sec - exiting 01:49:27 (5796): No heartbeat from core client for 30 sec - exiting 01:49:28 (5796): No heartbeat from core client for 30 sec - exiting 01:49:29 (5796): No heartbeat from core client for 30 sec - exiting 01:49:30 (5796): No heartbeat from core client for 30 sec - exiting 01:49:31 (5796): No heartbeat from core client for 30 sec - exiting 01:49:33 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:49:34 (5796): No heartbeat from core client for 30 sec - exiting 01:49:35 (5796): No heartbeat from core client for 30 sec - exiting 01:49:36 (5796): No heartbeat from core client for 30 sec - exiting 01:49:37 (5796): No heartbeat from core client for 30 sec - exiting 01:49:38 (5796): No heartbeat from core client for 30 sec - exiting 01:54:06 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:44 (2400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:45 (2400): No heartbeat from core client for 30 sec - exiting 01:55:47 (2400): No heartbeat from core client for 30 sec - exiting 02:01:14 (6312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:01:18 (6312): No heartbeat from core client for 30 sec - exiting 02:01:19 (6312): No heartbeat from core client for 30 sec - exiting 02:01:20 (6312): No heartbeat from core client for 30 sec - exiting 02:03:53 (2276): No heartbeat from core client for 30 sec - exiting 02:03:54 (2276): No heartbeat from core client for 30 sec - exiting 02:03:55 (2276): No heartbeat from core client for 30 sec - exiting 02:03:56 (2276): No heartbeat from core client for 30 sec - exiting 02:03:57 (2276): No heartbeat from core client for 30 sec - exiting 02:03:59 (2276): No heartbeat from core client for 30 sec - exiting 02:04:00 (2276): No heartbeat from core client for 30 sec - exiting 02:04:01 (2276): No heartbeat from core client for 30 sec - exiting 02:04:02 (2276): No heartbeat from core client for 30 sec - exiting 02:04:03 (2276): No heartbeat from core client for 30 sec - exiting 02:04:04 (2276): No heartbeat from core client for 30 sec - exiting 02:04:05 (2276): No heartbeat from core client for 30 sec - exiting 02:04:06 (2276): No heartbeat from core client for 30 sec - exiting 02:04:07 (2276): No heartbeat from core client for 30 sec - exiting 02:04:08 (2276): No heartbeat from core client for 30 sec - exiting 02:04:09 (2276): No heartbeat from core client for 30 sec - exiting 02:04:11 (2276): No heartbeat from core client for 30 sec - exiting 02:04:12 (2276): No heartbeat from core client for 30 sec - exiting 02:04:13 (2276): No heartbeat from core client for 30 sec - exiting 02:04:14 (2276): No heartbeat from core client for 30 sec - exiting 02:04:15 (2276): No heartbeat from core client for 30 sec - exiting 02:04:16 (2276): No heartbeat from core client for 30 sec - exiting 02:04:17 (2276): No heartbeat from core client for 30 sec - exiting 02:04:18 (2276): No heartbeat from core client for 30 sec - exiting 02:04:19 (2276): No heartbeat from core client for 30 sec - exiting 02:04:20 (2276): No heartbeat from core client for 30 sec - exiting 02:04:21 (2276): No heartbeat from core client for 30 sec - exiting 02:04:23 (2276): No heartbeat from core client for 30 sec - exiting 02:04:24 (2276): No heartbeat from core client for 30 sec - exiting 02:04:25 (2276): No heartbeat from core client for 30 sec - exiting 02:04:26 (2276): No heartbeat from core client for 30 sec - exiting 02:04:27 (2276): No heartbeat from core client for 30 sec - exiting 02:04:28 (2276): No heartbeat from core client for 30 sec - exiting 02:04:29 (2276): No heartbeat from core client for 30 sec - exiting 02:04:30 (2276): No heartbeat from core client for 30 sec - exiting 02:04:31 (2276): No heartbeat from core client for 30 sec - exiting 02:04:32 (2276): No heartbeat from core client for 30 sec - exiting 02:04:33 (2276): No heartbeat from core client for 30 sec - exiting 02:04:35 (2276): No heartbeat from core client for 30 sec - exiting 02:04:36 (2276): No heartbeat from core client for 30 sec - exiting 02:04:37 (2276): No heartbeat from core client for 30 sec - exiting 02:04:38 (2276): No heartbeat from core client for 30 sec - exiting 02:04:39 (2276): No heartbeat from core client for 30 sec - exiting 02:04:40 (2276): No heartbeat from core client for 30 sec - exiting 02:04:41 (2276): No heartbeat from core client for 30 sec - exiting 02:04:42 (2276): No heartbeat from core client for 30 sec - exiting 02:04:43 (2276): No heartbeat from core client for 30 sec - exiting 02:04:44 (2276): No heartbeat from core client for 30 sec - exiting 02:04:46 (2276): No heartbeat from core client for 30 sec - exiting 02:04:47 (2276): No heartbeat from core client for 30 sec - exiting 02:04:48 (2276): No heartbeat from core client for 30 sec - exiting 02:04:49 (2276): No heartbeat from core client for 30 sec - exiting 02:04:50 (2276): No heartbeat from core client for 30 sec - exiting 02:04:51 (2276): No heartbeat from core client for 30 sec - exiting 02:04:52 (2276): No heartbeat from core client for 30 sec - exiting 02:04:53 (2276): No heartbeat from core client for 30 sec - exiting 02:04:54 (2276): No heartbeat from core client for 30 sec - exiting 02:04:55 (2276): No heartbeat from core client for 30 sec - exiting 02:04:56 (2276): No heartbeat from core client for 30 sec - exiting 02:04:58 (2276): No heartbeat from core client for 30 sec - exiting 02:04:59 (2276): No heartbeat from core client for 30 sec - exiting 02:05:00 (2276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:05:01 (2276): No heartbeat from core client for 30 sec - exiting 02:05:02 (2276): No heartbeat from core client for 30 sec - exiting 02:09:00 (2976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:09:43 (5920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:14:32 (2904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:14:35 (2904): No heartbeat from core client for 30 sec - exiting 02:14:36 (2904): No heartbeat from core client for 30 sec - exiting 02:14:37 (2904): No heartbeat from core client for 30 sec - exiting 02:14:38 (2904): No heartbeat from core client for 30 sec - exiting 02:15:51 (3660): No heartbeat from core client for 30 sec - exiting 02:15:52 (3660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:54 (3660): No heartbeat from core client for 30 sec - exiting 02:15:55 (3660): No heartbeat from core client for 30 sec - exiting 02:15:56 (3660): No heartbeat from core client for 30 sec - exiting 02:15:57 (3660): No heartbeat from core client for 30 sec - exiting 02:15:58 (3660): No heartbeat from core client for 30 sec - exiting 02:15:59 (3660): No heartbeat from core client for 30 sec - exiting 02:23:14 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:23:15 (3476): No heartbeat from core client for 30 sec - exiting 02:32:35 (2868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:32:37 (2868): No heartbeat from core client for 30 sec - exiting 02:35:47 (1100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:40:41 (2428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:40:44 (2428): No heartbeat from core client for 30 sec - exiting 02:40:45 (2428): No heartbeat from core client for 30 sec - exiting 02:40:46 (2428): No heartbeat from core client for 30 sec - exiting 02:40:47 (2428): No heartbeat from core client for 30 sec - exiting 02:50:42 (7596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:50:44 (7596): No heartbeat from core client for 30 sec - exiting 02:50:45 (7596): No heartbeat from core client for 30 sec - exiting 02:54:13 (5572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:00:43 (3680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:00:45 (3680): No heartbeat from core client for 30 sec - exiting 03:00:46 (3680): No heartbeat from core client for 30 sec - exiting 03:10:56 (3064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:10:58 (3064): No heartbeat from core client for 30 sec - exiting 03:10:59 (3064): No heartbeat from core client for 30 sec - exiting 03:11:00 (3064): No heartbeat from core client for 30 sec - exiting 03:11:01 (3064): No heartbeat from core client for 30 sec - exiting 03:15:24 (7684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:21:18 (984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:21:20 (984): No heartbeat from core client for 30 sec - exiting 03:21:21 (984): No heartbeat from core client for 30 sec - exiting 03:21:23 (984): No heartbeat from core client for 30 sec - exiting 03:31:21 (5912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:31:23 (5912): No heartbeat from core client for 30 sec - exiting 03:31:24 (5912): No heartbeat from core client for 30 sec - exiting 03:41:23 (732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:41:27 (732): No heartbeat from core client for 30 sec - exiting 03:41:28 (732): No heartbeat from core client for 30 sec - exiting 03:41:29 (732): No heartbeat from core client for 30 sec - exiting 03:41:31 (732): No heartbeat from core client for 30 sec - exiting 03:51:25 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:51:30 (4304): No heartbeat from core client for 30 sec - exiting 03:51:31 (4304): No heartbeat from core client for 30 sec - exiting 03:51:32 (4304): No heartbeat from core client for 30 sec - exiting 03:51:34 (4304): No heartbeat from core client for 30 sec - exiting 03:51:35 (4304): No heartbeat from core client for 30 sec - exiting 03:51:36 (4304): No heartbeat from core client for 30 sec - exiting 03:51:37 (4304): No heartbeat from core client for 30 sec - exiting 03:51:38 (4304): No heartbeat from core client for 30 sec - exiting 04:01:55 (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:01:58 (4120): No heartbeat from core client for 30 sec - exiting 04:01:59 (4120): No heartbeat from core client for 30 sec - exiting 04:02:01 (4120): No heartbeat from core client for 30 sec - exiting 04:11:57 (2204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:11:59 (2204): No heartbeat from core client for 30 sec - exiting 04:12:00 (2204): No heartbeat from core client for 30 sec - exiting 04:21:57 (4516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:59 (4516): No heartbeat from core client for 30 sec - exiting 04:22:00 (4516): No heartbeat from core client for 30 sec - exiting 04:23:27 (7356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:59 (7024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:32:02 (7024): No heartbeat from core client for 30 sec - exiting 04:32:03 (7024): No heartbeat from core client for 30 sec - exiting 04:33:59 (2552): No heartbeat from core client for 30 sec - exiting 04:34:02 (2552): No heartbeat from core client for 30 sec - exiting 04:34:03 (2552): No heartbeat from core client for 30 sec - exiting 04:34:04 (2552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:34:05 (2552): No heartbeat from core client for 30 sec - exiting 04:34:06 (2552): No heartbeat from core client for 30 sec - exiting 04:34:07 (2552): No heartbeat from core client for 30 sec - exiting 04:38:26 (3440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:38:30 (3440): No heartbeat from core client for 30 sec - exiting 04:38:31 (3440): No heartbeat from core client for 30 sec - exiting 04:38:32 (3440): No heartbeat from core client for 30 sec - exiting 04:38:33 (3440): No heartbeat from core client for 30 sec - exiting 04:38:34 (3440): No heartbeat from core client for 30 sec - exiting 04:40:12 (5928): No heartbeat from core client for 30 sec - exiting 04:40:13 (5928): No heartbeat from core client for 30 sec - exiting 04:40:14 (5928): No heartbeat from core client for 30 sec - exiting 04:40:16 (5928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:44:49 (7500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:44:52 (7500): No heartbeat from core client for 30 sec - exiting 04:46:47 (5868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:00 (2244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:03 (2244): No heartbeat from core client for 30 sec - exiting 04:52:04 (2244): No heartbeat from core client for 30 sec - exiting 04:54:56 (1868): No heartbeat from core client for 30 sec - exiting 04:54:57 (1868): No heartbeat from core client for 30 sec - exiting 04:54:58 (1868): No heartbeat from core client for 30 sec - exiting 04:54:59 (1868): No heartbeat from core client for 30 sec - exiting 04:55:00 (1868): No heartbeat from core client for 30 sec - exiting 04:55:01 (1868): No heartbeat from core client for 30 sec - exiting 04:55:02 (1868): No heartbeat from core client for 30 sec - exiting 04:55:03 (1868): No heartbeat from core client for 30 sec - exiting 04:55:05 (1868): No heartbeat from core client for 30 sec - exiting 04:55:06 (1868): No heartbeat from core client for 30 sec - exiting 04:55:07 (1868): No heartbeat from core client for 30 sec - exiting 04:55:08 (1868): No heartbeat from core client for 30 sec - exiting 04:55:09 (1868): No heartbeat from core client for 30 sec - exiting 04:55:10 (1868): No heartbeat from core client for 30 sec - exiting 04:55:11 (1868): No heartbeat from core client for 30 sec - exiting 04:55:12 (1868): No heartbeat from core client for 30 sec - exiting 04:55:13 (1868): No heartbeat from core client for 30 sec - exiting 04:55:14 (1868): No heartbeat from core client for 30 sec - exiting 04:55:15 (1868): No heartbeat from core client for 30 sec - exiting 04:55:17 (1868): No heartbeat from core client for 30 sec - exiting 04:55:18 (1868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:00:49 (6864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:00:51 (6864): No heartbeat from core client for 30 sec - exiting 05:00:52 (6864): No heartbeat from core client for 30 sec - exiting 05:02:22 (4336): No heartbeat from core client for 30 sec - exiting 05:02:23 (4336): No heartbeat from core client for 30 sec - exiting 05:02:25 (4336): No heartbeat from core client for 30 sec - exiting 05:02:26 (4336): No heartbeat from core client for 30 sec - exiting 05:02:27 (4336): No heartbeat from core client for 30 sec - exiting 05:02:28 (4336): No heartbeat from core client for 30 sec - exiting 05:02:29 (4336): No heartbeat from core client for 30 sec - exiting 05:02:30 (4336): No heartbeat from core client for 30 sec - exiting 05:02:31 (4336): No heartbeat from core client for 30 sec - exiting 05:02:32 (4336): No heartbeat from core client for 30 sec - exiting 05:02:33 (4336): No heartbeat from core client for 30 sec - exiting 05:02:34 (4336): No heartbeat from core client for 30 sec - exiting 05:02:35 (4336): No heartbeat from core client for 30 sec - exiting 05:02:37 (4336): No heartbeat from core client for 30 sec - exiting 05:02:38 (4336): No heartbeat from core client for 30 sec - exiting 05:02:39 (4336): No heartbeat from core client for 30 sec - exiting 05:02:40 (4336): No heartbeat from core client for 30 sec - exiting 05:02:41 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:08:13 (5220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:08:18 (5220): No heartbeat from core client for 30 sec - exiting 05:08:19 (5220): No heartbeat from core client for 30 sec - exiting 05:08:20 (5220): No heartbeat from core client for 30 sec - exiting 05:08:21 (5220): No heartbeat from core client for 30 sec - exiting 05:17:15 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:17:18 (5184): No heartbeat from core client for 30 sec - exiting 05:17:19 (5184): No heartbeat from core client for 30 sec - exiting 05:17:20 (5184): No heartbeat from core client for 30 sec - exiting 05:19:31 (7320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:19:32 (7320): No heartbeat from core client for 30 sec - exiting 05:19:33 (7320): No heartbeat from core client for 30 sec - exiting 05:19:34 (7320): No heartbeat from core client for 30 sec - exiting 05:19:36 (7320): No heartbeat from core client for 30 sec - exiting 05:19:37 (7320): No heartbeat from core client for 30 sec - exiting 05:27:15 (6532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:27:18 (6532): No heartbeat from core client for 30 sec - exiting 05:27:20 (6532): No heartbeat from core client for 30 sec - exiting 05:27:22 (6532): No heartbeat from core client for 30 sec - exiting 05:29:08 (6420): No heartbeat from core client for 30 sec - exiting 05:29:09 (6420): No heartbeat from core client for 30 sec - exiting 05:29:10 (6420): No heartbeat from core client for 30 sec - exiting 05:29:11 (6420): No heartbeat from core client for 30 sec - exiting 05:29:12 (6420): No heartbeat from core client for 30 sec - exiting 05:29:13 (6420): No heartbeat from core client for 30 sec - exiting 05:29:15 (6420): No heartbeat from core client for 30 sec - exiting 05:29:16 (6420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:16 (3188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:18 (3188): No heartbeat from core client for 30 sec - exiting 05:37:19 (3188): No heartbeat from core client for 30 sec - exiting 05:38:56 (6964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:44:17 (6796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:44:19 (6796): No heartbeat from core client for 30 sec - exiting 05:45:39 (7308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:50:23 (6524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:50:25 (6524): No heartbeat from core client for 30 sec - exiting 05:50:26 (6524): No heartbeat from core client for 30 sec - exiting 05:50:27 (6524): No heartbeat from core client for 30 sec - exiting 05:51:48 (2132): No heartbeat from core client for 30 sec - exiting 05:51:49 (2132): No heartbeat from core client for 30 sec - exiting 05:51:50 (2132): No heartbeat from core client for 30 sec - exiting 05:51:51 (2132): No heartbeat from core client for 30 sec - exiting 05:51:52 (2132): No heartbeat from core client for 30 sec - exiting 05:51:54 (2132): No heartbeat from core client for 30 sec - exiting 05:51:55 (2132): No heartbeat from core client for 30 sec - exiting 05:51:56 (2132): No heartbeat from core client for 30 sec - exiting 05:51:57 (2132): No heartbeat from core client for 30 sec - exiting 05:51:58 (2132): No heartbeat from core client for 30 sec - exiting 05:51:59 (2132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:57:41 (992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:57:43 (992): No heartbeat from core client for 30 sec - exiting 05:57:44 (992): No heartbeat from core client for 30 sec - exiting 05:57:45 (992): No heartbeat from core client for 30 sec - exiting 05:59:58 (6696): No heartbeat from core client for 30 sec - exiting 05:59:59 (6696): No heartbeat from core client for 30 sec - exiting 06:00:00 (6696): No heartbeat from core client for 30 sec - exiting 06:00:01 (6696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:14 (3100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:17 (3100): No heartbeat from core client for 30 sec - exiting 22:05:18 (3100): No heartbeat from core client for 30 sec - exiting 22:07:05 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:11:34 (5124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:12:20 (7124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:16:49 (1040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:16:51 (1040): No heartbeat from core client for 30 sec - exiting 22:16:52 (1040): No heartbeat from core client for 30 sec - exiting 22:16:53 (1040): No heartbeat from core client for 30 sec - exiting 22:18:52 (4856): No heartbeat from core client for 30 sec - exiting 22:18:53 (4856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:18:54 (4856): No heartbeat from core client for 30 sec - exiting 22:18:55 (4856): No heartbeat from core client for 30 sec - exiting 22:24:18 (6192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:24:19 (6192): No heartbeat from core client for 30 sec - exiting 22:24:20 (6192): No heartbeat from core client for 30 sec - exiting 22:24:21 (6192): No heartbeat from core client for 30 sec - exiting 22:26:00 (2996): No heartbeat from core client for 30 sec - exiting 22:26:01 (2996): No heartbeat from core client for 30 sec - exiting 22:26:02 (2996): No heartbeat from core client for 30 sec - exiting 22:26:03 (2996): No heartbeat from core client for 30 sec - exiting 22:26:04 (2996): No heartbeat from core client for 30 sec - exiting 22:26:05 (2996): No heartbeat from core client for 30 sec - exiting 22:26:06 (2996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:07 (2996): No heartbeat from core client for 30 sec - exiting 22:31:38 (3988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:40 (3988): No heartbeat from core client for 30 sec - exiting 22:31:41 (3988): No heartbeat from core client for 30 sec - exiting 22:31:42 (3988): No heartbeat from core client for 30 sec - exiting 22:33:08 (2988): No heartbeat from core client for 30 sec - exiting 22:33:09 (2988): No heartbeat from core client for 30 sec - exiting 22:33:10 (2988): No heartbeat from core client for 30 sec - exiting 22:33:11 (2988): No heartbeat from core client for 30 sec - exiting 22:33:12 (2988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:13 (2988): No heartbeat from core client for 30 sec - exiting 22:33:14 (2988): No heartbeat from core client for 30 sec - exiting 22:38:25 (2188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:38:28 (2188): No heartbeat from core client for 30 sec - exiting 22:38:29 (2188): No heartbeat from core client for 30 sec - exiting 22:38:30 (2188): No heartbeat from core client for 30 sec - exiting 22:38:31 (2188): No heartbeat from core client for 30 sec - exiting 22:38:32 (2188): No heartbeat from core client for 30 sec - exiting 22:38:33 (2188): No heartbeat from core client for 30 sec - exiting 22:40:32 (4920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:40:33 (4920): No heartbeat from core client for 30 sec - exiting 22:40:34 (4920): No heartbeat from core client for 30 sec - exiting 22:46:53 (3524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:46:56 (3524): No heartbeat from core client for 30 sec - exiting 22:46:57 (3524): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7140, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7140, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7140, iMonCtr=1 Model crash detected, will try to restart... 22:47:59 (7140): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish 22:48:00 (7140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2692, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2692, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2692, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 22:54:11 (8092): No heartbeat from core client for 30 sec - exiting 22:54:13 (8092): No heartbeat from core client for 30 sec - exiting 22:54:14 (8092): No heartbeat from core client for 30 sec - exiting 22:54:15 (8092): No heartbeat from core client for 30 sec - exiting 22:54:16 (8092): No heartbeat from core client for 30 sec - exiting 22:54:17 (8092): No heartbeat from core client for 30 sec - exiting 22:54:18 (8092): No heartbeat from core client for 30 sec - exiting 22:54:20 (8092): No heartbeat from core client for 30 sec - exiting 22:54:21 (8092): No heartbeat from core client for 30 sec - exiting 22:54:22 (8092): No heartbeat from core client for 30 sec - exiting 22:54:23 (8092): No heartbeat from core client for 30 sec - exiting 22:54:24 (8092): No heartbeat from core client for 30 sec - exiting 22:54:25 (8092): No heartbeat from core client for 30 sec - exiting 22:54:26 (8092): No heartbeat from core client for 30 sec - exiting 22:54:27 (8092): No heartbeat from core client for 30 sec - exiting 22:54:28 (8092): No heartbeat from core client for 30 sec - exiting 22:54:29 (8092): No heartbeat from core client for 30 sec - exiting 22:54:30 (8092): No heartbeat from core client for 30 sec - exiting 22:54:32 (8092): No heartbeat from core client for 30 sec - exiting 22:54:33 (8092): No heartbeat from core client for 30 sec - exiting 22:54:34 (8092): No heartbeat from core client for 30 sec - exiting 22:54:35 (8092): No heartbeat from core client for 30 sec - exiting 22:54:36 (8092): No heartbeat from core client for 30 sec - exiting 22:54:37 (8092): No heartbeat from core client for 30 sec - exiting 22:54:38 (8092): No heartbeat from core client for 30 sec - exiting 22:54:39 (8092): No heartbeat from core client for 30 sec - exiting 22:54:40 (8092): No heartbeat from core client for 30 sec - exiting 22:54:41 (8092): No heartbeat from core client for 30 sec - exiting 22:54:42 (8092): No heartbeat from core client for 30 sec - exiting 22:54:44 (8092): No heartbeat from core client for 30 sec - exiting 22:54:45 (8092): No heartbeat from core client for 30 sec - exiting 22:54:46 (8092): No heartbeat from core client for 30 sec - exiting 22:54:47 (8092): No heartbeat from core client for 30 sec - exiting 22:54:48 (8092): No heartbeat from core client for 30 sec - exiting 22:54:49 (8092): No heartbeat from core client for 30 sec - exiting 22:54:50 (8092): No heartbeat from core client for 30 sec - exiting 22:54:51 (8092): No heartbeat from core client for 30 sec - exiting 22:54:52 (8092): No heartbeat from core client for 30 sec - exiting 22:54:53 (8092): No heartbeat from core client for 30 sec - exiting 22:54:54 (8092): No heartbeat from core client for 30 sec - exiting 22:54:56 (8092): No heartbeat from core client for 30 sec - exiting 22:54:57 (8092): No heartbeat from core client for 30 sec - exiting 22:54:58 (8092): No heartbeat from core client for 30 sec - exiting 22:54:59 (8092): No heartbeat from core client for 30 sec - exiting 22:55:00 (8092): No heartbeat from core client for 30 sec - exiting 22:55:01 (8092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:12 (4192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:15 (4192): No heartbeat from core client for 30 sec - exiting 23:06:08 (8168): No heartbeat from core client for 30 sec - exiting 23:06:09 (8168): No heartbeat from core client for 30 sec - exiting 23:06:10 (8168): No heartbeat from core client for 30 sec - exiting 23:06:11 (8168): No heartbeat from core client for 30 sec - exiting 23:06:12 (8168): No heartbeat from core client for 30 sec - exiting 23:06:14 (8168): No heartbeat from core client for 30 sec - exiting 23:06:15 (8168): No heartbeat from core client for 30 sec - exiting 23:06:16 (8168): No heartbeat from core client for 30 sec - exiting 23:06:17 (8168): No heartbeat from core client for 30 sec - exiting 23:06:18 (8168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:19 (8168): No heartbeat from core client for 30 sec - exiting 23:06:20 (8168): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1356, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1356, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1356, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1356, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1356, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1356, iMonCtr=1 Model crash detected, will try to restart... 23:08:57 (1356): No heartbeat from core client for 30 sec - exiting Sorry, too many model crashes! :-( Called boinc_finish 23:09:05 (1356): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Nov 2012 17:07:12	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	492,480	960,937	1.9512
25 Nov 2012 01:13:40	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	466,560	910,992	1.9526
24 Nov 2012 11:03:58	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	440,640	860,642	1.9532
23 Nov 2012 04:14:37	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	414,720	808,159	1.9487
21 Nov 2012 05:48:12	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	388,800	757,156	1.9474
19 Nov 2012 07:18:27	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	362,880	705,567	1.9444
18 Nov 2012 16:35:50	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	336,960	653,924	1.9407
18 Nov 2012 02:14:00	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	311,040	603,005	1.9387
17 Nov 2012 12:35:56	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	285,120	552,347	1.9372
16 Nov 2012 06:20:29	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	259,200	502,246	1.9377
14 Nov 2012 04:48:35	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	233,280	449,416	1.9265
12 Nov 2012 05:52:02	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	207,360	399,202	1.9252
11 Nov 2012 15:58:21	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	181,440	350,351	1.9309
11 Nov 2012 02:16:56	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	155,520	301,614	1.9394
10 Nov 2012 11:26:16	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	129,600	248,782	1.9196
07 Nov 2012 08:47:39	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	103,680	199,484	1.9240
05 Nov 2012 10:23:22	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	77,760	148,629	1.9114
04 Nov 2012 20:20:12	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	51,840	99,063	1.9109
04 Nov 2012 06:30:19	912411	15426137	hadcm3n_o4hm_2060_40_008239563_3	25,920	49,822	1.9221