Task 16588472

Name	hadcm3n_8az2_1980_40_008723641_0
Workunit	8869619
Created	23 Apr 2014, 13:03:05 UTC
Sent	3 May 2014, 16:33:54 UTC
Report deadline	3 Aug 2014, 0:01:05 UTC
Received	19 Aug 2014, 18:59:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1317427
Run time	7 days 22 hours 3 min 40 sec
CPU time	7 days 20 hours 39 min 20 sec
Validate state	Invalid
Credit	8,087.04
Device peak FLOPS	3.78 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.43</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> 18:49:00 (2587): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:01 (2587): No heartbeat from core client for 30 sec - exiting 18:49:02 (2587): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:43:07 (2434): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:43:08 (2434): No heartbeat from core client for 30 sec - exiting 12:43:09 (2434): No heartbeat from core client for 30 sec - exiting 12:43:10 (2434): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:47:56 (2516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:57 (2516): No heartbeat from core client for 30 sec - exiting 17:47:58 (2516): No heartbeat from core client for 30 sec - exiting 17:47:59 (2516): No heartbeat from core client for 30 sec - exiting 17:48:00 (2516): No heartbeat from core client for 30 sec - exiting 17:48:01 (2516): No heartbeat from core client for 30 sec - exiting 17:48:02 (2516): No heartbeat from core client for 30 sec - exiting 17:48:03 (2516): No heartbeat from core client for 30 sec - exiting 17:48:04 (2516): No heartbeat from core client for 30 sec - exiting 17:48:05 (2516): No heartbeat from core client for 30 sec - exiting 17:48:06 (2516): No heartbeat from core client for 30 sec - exiting 17:48:07 (2516): No heartbeat from core client for 30 sec - exiting 17:48:08 (2516): No heartbeat from core client for 30 sec - exiting 17:48:09 (2516): No heartbeat from core client for 30 sec - exiting 17:48:10 (2516): No heartbeat from core client for 30 sec - exiting 17:48:11 (2516): No heartbeat from core client for 30 sec - exiting 17:48:12 (2516): No heartbeat from core client for 30 sec - exiting 17:48:13 (2516): No heartbeat from core client for 30 sec - exiting 17:48:53 (3320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:36:53 (2462): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:48:10 (2571): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:48:11 (2571): No heartbeat from core client for 30 sec - exiting 17:48:12 (2571): No heartbeat from core client for 30 sec - exiting 17:48:13 (2571): No heartbeat from core client for 30 sec - exiting 17:48:14 (2571): No heartbeat from core client for 30 sec - exiting 17:48:15 (2571): No heartbeat from core client for 30 sec - exiting 17:48:16 (2571): No heartbeat from core client for 30 sec - exiting 17:48:17 (2571): No heartbeat from core client for 30 sec - exiting 17:48:18 (2571): No heartbeat from core client for 30 sec - exiting 17:48:19 (2571): No heartbeat from core client for 30 sec - exiting 17:48:20 (2571): No heartbeat from core client for 30 sec - exiting 17:48:21 (2571): No heartbeat from core client for 30 sec - exiting 17:48:22 (2571): No heartbeat from core client for 30 sec - exiting 17:48:23 (2571): No heartbeat from core client for 30 sec - exiting 17:48:24 (2571): No heartbeat from core client for 30 sec - exiting 17:48:25 (2571): No heartbeat from core client for 30 sec - exiting 17:48:26 (2571): No heartbeat from core client for 30 sec - exiting 17:48:27 (2571): No heartbeat from core client for 30 sec - exiting 17:48:28 (2571): No heartbeat from core client for 30 sec - exiting 17:48:29 (2571): No heartbeat from core client for 30 sec - exiting 17:48:30 (2571): No heartbeat from core client for 30 sec - exiting 17:48:31 (2571): No heartbeat from core client for 30 sec - exiting 17:48:32 (2571): No heartbeat from core client for 30 sec - exiting 17:48:33 (2571): No heartbeat from core client for 30 sec - exiting 17:48:34 (2571): No heartbeat from core client for 30 sec - exiting 17:48:35 (2571): No heartbeat from core client for 30 sec - exiting 17:48:36 (2571): No heartbeat from core client for 30 sec - exiting 17:48:37 (2571): No heartbeat from core client for 30 sec - exiting 17:48:38 (2571): No heartbeat from core client for 30 sec - exiting 17:48:39 (2571): No heartbeat from core client for 30 sec - exiting 17:48:40 (2571): No heartbeat from core client for 30 sec - exiting 17:48:41 (2571): No heartbeat from core client for 30 sec - exiting 17:48:42 (2571): No heartbeat from core client for 30 sec - exiting 17:50:59 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:00 (3352): No heartbeat from core client for 30 sec - exiting 17:51:01 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:32:30 (4267): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:31 (4267): No heartbeat from core client for 30 sec - exiting 20:32:32 (4267): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:56:54 (2408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:56 (2408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:50:51 (2452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:42:54 (2410): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:47:38 (3119): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:46:32 (2569): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:33 (2569): No heartbeat from core client for 30 sec - exiting 19:46:34 (2569): No heartbeat from core client for 30 sec - exiting 19:46:35 (2569): No heartbeat from core client for 30 sec - exiting 19:46:36 (2569): No heartbeat from core client for 30 sec - exiting 19:46:37 (2569): No heartbeat from core client for 30 sec - exiting 19:46:38 (2569): No heartbeat from core client for 30 sec - exiting 19:46:39 (2569): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:01:24 (2533): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:01:25 (2533): No heartbeat from core client for 30 sec - exiting 22:01:26 (2533): No heartbeat from core client for 30 sec - exiting 22:05:02 (3223): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:03 (3223): No heartbeat from core client for 30 sec - exiting 22:05:04 (3223): No heartbeat from core client for 30 sec - exiting 22:05:05 (3223): No heartbeat from core client for 30 sec - exiting 22:05:06 (3223): No heartbeat from core client for 30 sec - exiting 22:05:07 (3223): No heartbeat from core client for 30 sec - exiting 22:05:08 (3223): No heartbeat from core client for 30 sec - exiting 22:05:09 (3223): No heartbeat from core client for 30 sec - exiting 22:05:10 (3223): No heartbeat from core client for 30 sec - exiting 22:05:11 (3223): No heartbeat from core client for 30 sec - exiting 22:05:12 (3223): No heartbeat from core client for 30 sec - exiting 22:05:13 (3223): No heartbeat from core client for 30 sec - exiting 22:05:14 (3223): No heartbeat from core client for 30 sec - exiting 22:05:15 (3223): No heartbeat from core client for 30 sec - exiting 22:05:16 (3223): No heartbeat from core client for 30 sec - exiting 22:05:17 (3223): No heartbeat from core client for 30 sec - exiting 22:05:18 (3223): No heartbeat from core client for 30 sec - exiting 22:05:19 (3223): No heartbeat from core client for 30 sec - exiting 22:05:20 (3223): No heartbeat from core client for 30 sec - exiting 22:06:01 (3664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:47:59 (2446): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:48:00 (2446): No heartbeat from core client for 30 sec - exiting 09:48:01 (2446): No heartbeat from core client for 30 sec - exiting 09:48:02 (2446): No heartbeat from core client for 30 sec - exiting 09:48:03 (2446): No heartbeat from core client for 30 sec - exiting 09:48:04 (2446): No heartbeat from core client for 30 sec - exiting 09:48:05 (2446): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:48:30 (2673): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:48:32 (2673): No heartbeat from core client for 30 sec - exiting 17:48:33 (2673): No heartbeat from core client for 30 sec - exiting 17:48:34 (2673): No heartbeat from core client for 30 sec - exiting 17:48:35 (2673): No heartbeat from core client for 30 sec - exiting 17:48:36 (2673): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:53:22 (2438): No heartbeat from core client for 30 sec - exiting 10:53:23 (2438): No heartbeat from core client for 30 sec - exiting 10:53:24 (2438): No heartbeat from core client for 30 sec - exiting 10:53:25 (2438): No heartbeat from core client for 30 sec - exiting 10:53:26 (2438): No heartbeat from core client for 30 sec - exiting 10:53:27 (2438): No heartbeat from core client for 30 sec - exiting 10:53:28 (2438): No heartbeat from core client for 30 sec - exiting 10:53:29 (2438): No heartbeat from core client for 30 sec - exiting 10:53:30 (2438): No heartbeat from core client for 30 sec - exiting 10:53:31 (2438): No heartbeat from core client for 30 sec - exiting 10:53:32 (2438): No heartbeat from core client for 30 sec - exiting 10:53:33 (2438): No heartbeat from core client for 30 sec - exiting 10:53:34 (2438): No heartbeat from core client for 30 sec - exiting 10:53:35 (2438): No heartbeat from core client for 30 sec - exiting 10:53:36 (2438): No heartbeat from core client for 30 sec - exiting 10:53:37 (2438): No heartbeat from core client for 30 sec - exiting 10:53:38 (2438): No heartbeat from core client for 30 sec - exiting 10:53:39 (2438): No heartbeat from core client for 30 sec - exiting 10:53:40 (2438): No heartbeat from core client for 30 sec - exiting 10:53:41 (2438): No heartbeat from core client for 30 sec - exiting 10:53:42 (2438): No heartbeat from core client for 30 sec - exiting 10:53:43 (2438): No heartbeat from core client for 30 sec - exiting 10:53:44 (2438): No heartbeat from core client for 30 sec - exiting 10:53:45 (2438): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:01:02 (2588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:54:07 (2558): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:54:08 (2558): No heartbeat from core client for 30 sec - exiting 18:54:09 (2558): No heartbeat from core client for 30 sec - exiting 18:54:10 (2558): No heartbeat from core client for 30 sec - exiting 18:54:11 (2558): No heartbeat from core client for 30 sec - exiting 18:54:12 (2558): No heartbeat from core client for 30 sec - exiting 18:54:13 (2558): No heartbeat from core client for 30 sec - exiting 18:54:14 (2558): No heartbeat from core client for 30 sec - exiting 18:54:15 (2558): No heartbeat from core client for 30 sec - exiting 18:54:16 (2558): No heartbeat from core client for 30 sec - exiting 18:54:17 (2558): No heartbeat from core client for 30 sec - exiting 18:54:18 (2558): No heartbeat from core client for 30 sec - exiting 18:54:19 (2558): No heartbeat from core client for 30 sec - exiting 20:03:50 (4085): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:51 (4085): No heartbeat from core client for 30 sec - exiting 20:03:52 (4085): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:26:51 (2564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:53 (2564): No heartbeat from core client for 30 sec - exiting 20:26:54 (2564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:01:40 (2523): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:01:41 (2523): No heartbeat from core client for 30 sec - exiting 12:01:42 (2523): No heartbeat from core client for 30 sec - exiting 12:01:43 (2523): No heartbeat from core client for 30 sec - exiting 12:01:44 (2523): No heartbeat from core client for 30 sec - exiting 12:01:45 (2523): No heartbeat from core client for 30 sec - exiting 12:01:46 (2523): No heartbeat from core client for 30 sec - exiting 12:01:47 (2523): No heartbeat from core client for 30 sec - exiting 12:01:48 (2523): No heartbeat from core client for 30 sec - exiting 12:01:49 (2523): No heartbeat from core client for 30 sec - exiting 12:01:50 (2523): No heartbeat from core client for 30 sec - exiting 12:01:51 (2523): No heartbeat from core client for 30 sec - exiting 12:01:52 (2523): No heartbeat from core client for 30 sec - exiting 12:01:53 (2523): No heartbeat from core client for 30 sec - exiting 12:01:54 (2523): No heartbeat from core client for 30 sec - exiting 12:01:55 (2523): No heartbeat from core client for 30 sec - exiting 12:01:56 (2523): No heartbeat from core client for 30 sec - exiting 18:31:41 (3088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:21:23 (2563): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:24 (2563): No heartbeat from core client for 30 sec - exiting 22:21:25 (2563): No heartbeat from core client for 30 sec - exiting 22:21:26 (2563): No heartbeat from core client for 30 sec - exiting 22:21:27 (2563): No heartbeat from core client for 30 sec - exiting 22:21:28 (2563): No heartbeat from core client for 30 sec - exiting 22:21:29 (2563): No heartbeat from core client for 30 sec - exiting 22:21:30 (2563): No heartbeat from core client for 30 sec - exiting 23:02:55 (2627): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:02:56 (2627): No heartbeat from core client for 30 sec - exiting 23:02:57 (2627): No heartbeat from core client for 30 sec - exiting 23:02:58 (2627): No heartbeat from core client for 30 sec - exiting 23:02:59 (2627): No heartbeat from core client for 30 sec - exiting 23:18:56 (2772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:18:57 (2772): No heartbeat from core client for 30 sec - exiting 23:18:58 (2772): No heartbeat from core client for 30 sec - exiting 23:18:59 (2772): No heartbeat from core client for 30 sec - exiting 23:19:00 (2772): No heartbeat from core client for 30 sec - exiting 23:19:01 (2772): No heartbeat from core client for 30 sec - exiting 23:19:02 (2772): No heartbeat from core client for 30 sec - exiting 23:19:03 (2772): No heartbeat from core client for 30 sec - exiting 23:19:04 (2772): No heartbeat from core client for 30 sec - exiting 23:19:05 (2772): No heartbeat from core client for 30 sec - exiting 23:19:06 (2772): No heartbeat from core client for 30 sec - exiting 23:19:07 (2772): No heartbeat from core client for 30 sec - exiting 23:19:08 (2772): No heartbeat from core client for 30 sec - exiting 23:19:09 (2772): No heartbeat from core client for 30 sec - exiting 23:19:10 (2772): No heartbeat from core client for 30 sec - exiting 23:19:11 (2772): No heartbeat from core client for 30 sec - exiting 23:19:12 (2772): No heartbeat from core client for 30 sec - exiting 23:19:13 (2772): No heartbeat from core client for 30 sec - exiting 23:19:14 (2772): No heartbeat from core client for 30 sec - exiting 23:19:15 (2772): No heartbeat from core client for 30 sec - exiting 23:19:16 (2772): No heartbeat from core client for 30 sec - exiting 23:19:17 (2772): No heartbeat from core client for 30 sec - exiting 23:19:18 (2772): No heartbeat from core client for 30 sec - exiting 23:19:19 (2772): No heartbeat from core client for 30 sec - exiting 23:19:20 (2772): No heartbeat from core client for 30 sec - exiting 23:19:21 (2772): No heartbeat from core client for 30 sec - exiting 23:19:22 (2772): No heartbeat from core client for 30 sec - exiting 23:19:23 (2772): No heartbeat from core client for 30 sec - exiting 23:19:24 (2772): No heartbeat from core client for 30 sec - exiting 23:19:25 (2772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 21:42:57 (2424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:58 (2424): No heartbeat from core client for 30 sec - exiting 21:43:00 (2424): No heartbeat from core client for 30 sec - exiting 21:43:01 (2424): No heartbeat from core client for 30 sec - exiting 21:43:02 (2424): No heartbeat from core client for 30 sec - exiting 21:47:26 (2564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:27 (2564): No heartbeat from core client for 30 sec - exiting 21:47:28 (2564): No heartbeat from core client for 30 sec - exiting 21:47:29 (2564): No heartbeat from core client for 30 sec - exiting 21:47:30 (2564): No heartbeat from core client for 30 sec - exiting 21:47:31 (2564): No heartbeat from core client for 30 sec - exiting 21:47:32 (2564): No heartbeat from core client for 30 sec - exiting 21:47:33 (2564): No heartbeat from core client for 30 sec - exiting 21:47:34 (2564): No heartbeat from core client for 30 sec - exiting 21:47:35 (2564): No heartbeat from core client for 30 sec - exiting 21:47:36 (2564): No heartbeat from core client for 30 sec - exiting 21:47:37 (2564): No heartbeat from core client for 30 sec - exiting 21:47:38 (2564): No heartbeat from core client for 30 sec - exiting 21:47:39 (2564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 09:00:00 (2420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:00:02 (2420): No heartbeat from core client for 30 sec - exiting 09:00:03 (2420): No heartbeat from core client for 30 sec - exiting 09:00:04 (2420): No heartbeat from core client for 30 sec - exiting 09:00:05 (2420): No heartbeat from core client for 30 sec - exiting 09:00:06 (2420): No heartbeat from core client for 30 sec - exiting 09:00:07 (2420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:07:02 (2425): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:07:40 (3801): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:19:23 (3902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:19:24 (3902): No heartbeat from core client for 30 sec - exiting 20:19:25 (3902): No heartbeat from core client for 30 sec - exiting 20:19:26 (3902): No heartbeat from core client for 30 sec - exiting 20:19:27 (3902): No heartbeat from core client for 30 sec - exiting 20:19:28 (3902): No heartbeat from core client for 30 sec - exiting 20:19:29 (3902): No heartbeat from core client for 30 sec - exiting 20:19:30 (3902): No heartbeat from core client for 30 sec - exiting 20:19:31 (3902): No heartbeat from core client for 30 sec - exiting 20:19:32 (3902): No heartbeat from core client for 30 sec - exiting 20:19:33 (3902): No heartbeat from core client for 30 sec - exiting 20:19:34 (3902): No heartbeat from core client for 30 sec - exiting 20:19:35 (3902): No heartbeat from core client for 30 sec - exiting 20:19:36 (3902): No heartbeat from core client for 30 sec - exiting 20:19:37 (3902): No heartbeat from core client for 30 sec - exiting 20:19:38 (3902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:04:22 (2414): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:04:23 (2414): No heartbeat from core client for 30 sec - exiting 20:04:24 (2414): No heartbeat from core client for 30 sec - exiting 20:04:25 (2414): No heartbeat from core client for 30 sec - exiting 20:04:26 (2414): No heartbeat from core client for 30 sec - exiting 20:04:27 (2414): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf76fb400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf76fb430] /lib/libc.so.6(gsignal+0x46)[0xf750e6a6] /lib/libc.so.6(abort+0x143)[0xf750fe33] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf74f99d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3707, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7760400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7760430] /lib/libc.so.6(gsignal+0x46)[0xf75736a6] /lib/libc.so.6(abort+0x143)[0xf7574e33] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf755e9d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3707, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf76f2400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf76f2430] /lib/libc.so.6(gsignal+0x46)[0xf75056a6] /lib/libc.so.6(abort+0x143)[0xf7506e33] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf74f09d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3707, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7708400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf7708430] /lib/libc.so.6(gsignal+0x46)[0xf751b6a6] /lib/libc.so.6(abort+0x143)[0xf751ce33] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf75069d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3707, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf776f400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf776f430] /lib/libc.so.6(gsignal+0x46)[0xf75826a6] /lib/libc.so.6(abort+0x143)[0xf7583e33] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf756d9d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3707, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77a3400] linux-gate.so.1(__kernel_vsyscall+0x10)[0xf77a3430] /lib/libc.so.6(gsignal+0x46)[0xf75b66a6] /lib/libc.so.6(abort+0x143)[0xf75b7e33] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf75a19d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3707, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Aug 2014 19:32:43	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	673,920	669,433	0.9933
17 Aug 2014 16:00:24	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	648,000	643,588	0.9932
17 Aug 2014 08:56:14	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	622,080	618,063	0.9935
16 Aug 2014 14:09:32	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	596,160	592,529	0.9939
07 Aug 2014 19:47:09	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	570,240	566,761	0.9939
07 Aug 2014 19:47:08	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	544,320	540,656	0.9933
07 Aug 2014 19:47:08	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	518,400	514,610	0.9927
23 Jul 2014 18:21:57	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	492,480	488,828	0.9926
20 Jul 2014 15:20:50	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	466,560	463,142	0.9927
19 Jul 2014 18:10:36	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	440,640	437,285	0.9924
19 Jul 2014 11:06:52	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	414,720	411,811	0.9930
17 Jul 2014 18:47:07	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	388,800	385,848	0.9924
13 Jul 2014 18:06:34	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	362,880	360,075	0.9923
12 Jul 2014 16:39:20	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	336,960	334,371	0.9923
10 Jul 2014 18:39:01	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	311,040	308,456	0.9917
29 Jun 2014 14:32:18	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	285,120	282,374	0.9904
26 Jun 2014 20:03:12	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	259,200	256,252	0.9886
17 Jun 2014 18:47:20	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	233,280	230,614	0.9886
10 Jun 2014 09:03:19	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	207,360	204,853	0.9879
10 Jun 2014 09:01:54	1317427	16588472	hadcm3n_8az2_1980_40_008723641_0	181,440	179,352	0.9885