Task 12829737

Name	hadcm3n_p5nd_1900_40_007224905_0
Workunit	7423145
Created	26 Apr 2011, 15:33:26 UTC
Sent	28 Apr 2011, 0:39:28 UTC
Report deadline	28 Jul 2011, 8:06:39 UTC
Received	8 May 2011, 3:13:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	845875
Run time	6 days 19 hours 41 min 21 sec
CPU time	3 days 3 hours 50 min 59 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	1.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:35:22 (20147): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:35:24 (20147): No heartbeat from core client for 30 sec - exiting 18:35:25 (20147): No heartbeat from core client for 30 sec - exiting 18:35:26 (20147): No heartbeat from core client for 30 sec - exiting 18:35:27 (20147): No heartbeat from core client for 30 sec - exiting 18:35:28 (20147): No heartbeat from core client for 30 sec - exiting 18:35:29 (20147): No heartbeat from core client for 30 sec - exiting 18:35:30 (20147): No heartbeat from core client for 30 sec - exiting 18:35:31 (20147): No heartbeat from core client for 30 sec - exiting 18:35:32 (20147): No heartbeat from core client for 30 sec - exiting 18:35:33 (20147): No heartbeat from core client for 30 sec - exiting 18:35:34 (20147): No heartbeat from core client for 30 sec - exiting 18:35:35 (20147): No heartbeat from core client for 30 sec - exiting 18:35:36 (20147): No heartbeat from core client for 30 sec - exiting 18:35:37 (20147): No heartbeat from core client for 30 sec - exiting 18:35:38 (20147): No heartbeat from core client for 30 sec - exiting 18:35:39 (20147): No heartbeat from core client for 30 sec - exiting 18:35:40 (20147): No heartbeat from core client for 30 sec - exiting 00:48:09 (23408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:05:40 (8293): No heartbeat from core client for 30 sec - exiting 02:27:23 (8293): No heartbeat from core client for 30 sec - exiting 02:27:24 (8293): No heartbeat from core client for 30 sec - exiting 02:27:25 (8293): No heartbeat from core client for 30 sec - exiting 02:27:26 (8293): No heartbeat from core client for 30 sec - exiting 02:27:27 (8293): No heartbeat from core client for 30 sec - exiting 02:27:28 (8293): No heartbeat from core client for 30 sec - exiting 02:27:29 (8293): No heartbeat from core client for 30 sec - exiting 02:27:30 (8293): No heartbeat from core client for 30 sec - exiting 02:27:31 (8293): No heartbeat from core client for 30 sec - exiting 02:27:32 (8293): No heartbeat from core client for 30 sec - exiting 02:27:33 (8293): No heartbeat from core client for 30 sec - exiting 02:27:34 (8293): No heartbeat from core client for 30 sec - exiting 02:27:35 (8293): No heartbeat from core client for 30 sec - exiting 02:27:36 (8293): No heartbeat from core client for 30 sec - exiting 02:27:37 (8293): No heartbeat from core client for 30 sec - exiting 02:27:38 (8293): No heartbeat from core client for 30 sec - exiting 02:27:39 (8293): No heartbeat from core client for 30 sec - exiting 02:27:40 (8293): No heartbeat from core client for 30 sec - exiting 02:27:41 (8293): No heartbeat from core client for 30 sec - exiting 02:27:42 (8293): No heartbeat from core client for 30 sec - exiting 02:27:43 (8293): No heartbeat from core client for 30 sec - exiting 02:27:44 (8293): No heartbeat from core client for 30 sec - exiting 02:27:45 (8293): No heartbeat from core client for 30 sec - exiting 02:28:25 (8293): No heartbeat from core client for 30 sec - exiting 02:28:26 (8293): No heartbeat from core client for 30 sec - exiting 02:28:27 (8293): No heartbeat from core client for 30 sec - exiting 02:28:28 (8293): No heartbeat from core client for 30 sec - exiting 02:28:29 (8293): No heartbeat from core client for 30 sec - exiting 02:58:36 (8293): No heartbeat from core client for 30 sec - exiting 03:16:41 (8293): No heartbeat from core client for 30 sec - exiting 03:28:59 (8293): No heartbeat from core client for 30 sec - exiting 04:01:36 (8293): No heartbeat from core client for 30 sec - exiting 04:18:24 (8293): No heartbeat from core client for 30 sec - exiting 04:26:56 (8293): No heartbeat from core client for 30 sec - exiting 05:24:49 (8293): No heartbeat from core client for 30 sec - exiting 05:46:25 (8293): No heartbeat from core client for 30 sec - exiting 05:48:42 (8293): No heartbeat from core client for 30 sec - exiting 05:59:33 (8293): No heartbeat from core client for 30 sec - exiting 06:05:16 (8293): No heartbeat from core client for 30 sec - exiting 06:39:59 (8293): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:46 (9767): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:47 (9767): No heartbeat from core client for 30 sec - exiting 07:12:41 (9998): No heartbeat from core client for 30 sec - exiting 07:12:42 (9998): No heartbeat from core client for 30 sec - exiting 07:12:43 (9998): No heartbeat from core client for 30 sec - exiting 07:12:44 (9998): No heartbeat from core client for 30 sec - exiting 07:12:45 (9998): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:19:17 (10037): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:34:04 (10092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 16:58:26 (7148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:30:27 (18220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:30:28 (18220): No heartbeat from core client for 30 sec - exiting 20:30:29 (18220): No heartbeat from core client for 30 sec - exiting 20:30:30 (18220): No heartbeat from core client for 30 sec - exiting 20:30:31 (18220): No heartbeat from core client for 30 sec - exiting 20:30:32 (18220): No heartbeat from core client for 30 sec - exiting 20:30:33 (18220): No heartbeat from core client for 30 sec - exiting 20:30:34 (18220): No heartbeat from core client for 30 sec - exiting 20:30:35 (18220): No heartbeat from core client for 30 sec - exiting 20:30:36 (18220): No heartbeat from core client for 30 sec - exiting 20:30:37 (18220): No heartbeat from core client for 30 sec - exiting 20:30:38 (18220): No heartbeat from core client for 30 sec - exiting 20:30:39 (18220): No heartbeat from core client for 30 sec - exiting 20:30:40 (18220): No heartbeat from core client for 30 sec - exiting 20:30:41 (18220): No heartbeat from core client for 30 sec - exiting 20:30:42 (18220): No heartbeat from core client for 30 sec - exiting 20:30:43 (18220): No heartbeat from core client for 30 sec - exiting 20:30:44 (18220): No heartbeat from core client for 30 sec - exiting 20:30:45 (18220): No heartbeat from core client for 30 sec - exiting 20:30:46 (18220): No heartbeat from core client for 30 sec - exiting 20:30:47 (18220): No heartbeat from core client for 30 sec - exiting 20:30:48 (18220): No heartbeat from core client for 30 sec - exiting 20:30:49 (18220): No heartbeat from core client for 30 sec - exiting 20:30:50 (18220): No heartbeat from core client for 30 sec - exiting 20:30:51 (18220): No heartbeat from core client for 30 sec - exiting 20:30:52 (18220): No heartbeat from core client for 30 sec - exiting 20:30:53 (18220): No heartbeat from core client for 30 sec - exiting 20:30:54 (18220): No heartbeat from core client for 30 sec - exiting 20:30:55 (18220): No heartbeat from core client for 30 sec - exiting 20:30:56 (18220): No heartbeat from core client for 30 sec - exiting 20:30:57 (18220): No heartbeat from core client for 30 sec - exiting 20:30:58 (18220): No heartbeat from core client for 30 sec - exiting 20:30:59 (18220): No heartbeat from core client for 30 sec - exiting 20:31:00 (18220): No heartbeat from core client for 30 sec - exiting 20:31:01 (18220): No heartbeat from core client for 30 sec - exiting 20:31:02 (18220): No heartbeat from core client for 30 sec - exiting 20:34:34 (18730): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:34:35 (18730): No heartbeat from core client for 30 sec - exiting 20:34:36 (18730): No heartbeat from core client for 30 sec - exiting 20:34:37 (18730): No heartbeat from core client for 30 sec - exiting 20:34:38 (18730): No heartbeat from core client for 30 sec - exiting 20:34:39 (18730): No heartbeat from core client for 30 sec - exiting 20:34:40 (18730): No heartbeat from core client for 30 sec - exiting 20:34:41 (18730): No heartbeat from core client for 30 sec - exiting 20:34:42 (18730): No heartbeat from core client for 30 sec - exiting 20:34:43 (18730): No heartbeat from core client for 30 sec - exiting 20:34:44 (18730): No heartbeat from core client for 30 sec - exiting 20:34:45 (18730): No heartbeat from core client for 30 sec - exiting 20:34:46 (18730): No heartbeat from core client for 30 sec - exiting 20:34:47 (18730): No heartbeat from core client for 30 sec - exiting 20:34:48 (18730): No heartbeat from core client for 30 sec - exiting 20:34:49 (18730): No heartbeat from core client for 30 sec - exiting 20:34:50 (18730): No heartbeat from core client for 30 sec - exiting 20:34:51 (18730): No heartbeat from core client for 30 sec - exiting 20:47:49 (18759): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:50:42 (18766): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:52:58 (18777): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:56:50 (18788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:00:09 (18797): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:38 (18807): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:05:18 (18817): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:05:19 (18817): No heartbeat from core client for 30 sec - exiting 21:05:20 (18817): No heartbeat from core client for 30 sec - exiting 21:05:21 (18817): No heartbeat from core client for 30 sec - exiting 21:05:22 (18817): No heartbeat from core client for 30 sec - exiting 21:05:23 (18817): No heartbeat from core client for 30 sec - exiting 21:05:24 (18817): No heartbeat from core client for 30 sec - exiting 21:05:25 (18817): No heartbeat from core client for 30 sec - exiting 21:05:26 (18817): No heartbeat from core client for 30 sec - exiting 21:05:27 (18817): No heartbeat from core client for 30 sec - exiting 21:05:28 (18817): No heartbeat from core client for 30 sec - exiting 21:05:29 (18817): No heartbeat from core client for 30 sec - exiting 21:05:30 (18817): No heartbeat from core client for 30 sec - exiting 21:05:31 (18817): No heartbeat from core client for 30 sec - exiting 21:05:32 (18817): No heartbeat from core client for 30 sec - exiting 21:05:33 (18817): No heartbeat from core client for 30 sec - exiting 21:05:34 (18817): No heartbeat from core client for 30 sec - exiting 21:05:35 (18817): No heartbeat from core client for 30 sec - exiting 21:05:36 (18817): No heartbeat from core client for 30 sec - exiting 21:05:37 (18817): No heartbeat from core client for 30 sec - exiting 21:05:38 (18817): No heartbeat from core client for 30 sec - exiting 21:05:39 (18817): No heartbeat from core client for 30 sec - exiting 21:05:40 (18817): No heartbeat from core client for 30 sec - exiting 21:05:41 (18817): No heartbeat from core client for 30 sec - exiting 21:05:42 (18817): No heartbeat from core client for 30 sec - exiting 21:05:43 (18817): No heartbeat from core client for 30 sec - exiting 21:05:44 (18817): No heartbeat from core client for 30 sec - exiting 21:05:45 (18817): No heartbeat from core client for 30 sec - exiting 21:05:46 (18817): No heartbeat from core client for 30 sec - exiting 21:05:47 (18817): No heartbeat from core client for 30 sec - exiting 21:05:48 (18817): No heartbeat from core client for 30 sec - exiting 21:05:49 (18817): No heartbeat from core client for 30 sec - exiting 21:05:50 (18817): No heartbeat from core client for 30 sec - exiting 21:05:51 (18817): No heartbeat from core client for 30 sec - exiting 21:05:52 (18817): No heartbeat from core client for 30 sec - exiting 21:05:53 (18817): No heartbeat from core client for 30 sec - exiting 21:05:54 (18817): No heartbeat from core client for 30 sec - exiting 21:05:55 (18817): No heartbeat from core client for 30 sec - exiting 21:05:56 (18817): No heartbeat from core client for 30 sec - exiting 21:05:57 (18817): No heartbeat from core client for 30 sec - exiting 21:05:58 (18817): No heartbeat from core client for 30 sec - exiting 21:05:59 (18817): No heartbeat from core client for 30 sec - exiting 21:06:00 (18817): No heartbeat from core client for 30 sec - exiting 21:06:01 (18817): No heartbeat from core client for 30 sec - exiting 21:06:02 (18817): No heartbeat from core client for 30 sec - exiting 21:06:03 (18817): No heartbeat from core client for 30 sec - exiting 21:06:04 (18817): No heartbeat from core client for 30 sec - exiting 21:06:05 (18817): No heartbeat from core client for 30 sec - exiting 21:06:06 (18817): No heartbeat from core client for 30 sec - exiting 21:06:07 (18817): No heartbeat from core client for 30 sec - exiting 21:06:08 (18817): No heartbeat from core client for 30 sec - exiting 21:06:09 (18817): No heartbeat from core client for 30 sec - exiting 21:06:10 (18817): No heartbeat from core client for 30 sec - exiting 21:06:11 (18817): No heartbeat from core client for 30 sec - exiting 21:09:14 (18828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:46 (18852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:16:11 (18860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:11 (18894): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:04 (18907): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:12 (18912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:13 (18912): No heartbeat from core client for 30 sec - exiting 21:24:14 (18912): No heartbeat from core client for 30 sec - exiting 21:24:15 (18912): No heartbeat from core client for 30 sec - exiting 21:24:16 (18912): No heartbeat from core client for 30 sec - exiting 21:24:17 (18912): No heartbeat from core client for 30 sec - exiting 21:24:18 (18912): No heartbeat from core client for 30 sec - exiting 21:24:19 (18912): No heartbeat from core client for 30 sec - exiting 21:24:20 (18912): No heartbeat from core client for 30 sec - exiting 21:24:21 (18912): No heartbeat from core client for 30 sec - exiting 21:24:22 (18912): No heartbeat from core client for 30 sec - exiting 21:24:23 (18912): No heartbeat from core client for 30 sec - exiting 21:24:24 (18912): No heartbeat from core client for 30 sec - exiting 21:24:25 (18912): No heartbeat from core client for 30 sec - exiting 21:24:26 (18912): No heartbeat from core client for 30 sec - exiting 21:24:27 (18912): No heartbeat from core client for 30 sec - exiting 21:24:28 (18912): No heartbeat from core client for 30 sec - exiting 21:24:29 (18912): No heartbeat from core client for 30 sec - exiting 21:24:30 (18912): No heartbeat from core client for 30 sec - exiting 21:24:31 (18912): No heartbeat from core client for 30 sec - exiting 21:24:32 (18912): No heartbeat from core client for 30 sec - exiting 21:24:33 (18912): No heartbeat from core client for 30 sec - exiting 21:24:34 (18912): No heartbeat from core client for 30 sec - exiting 21:24:35 (18912): No heartbeat from core client for 30 sec - exiting 21:24:36 (18912): No heartbeat from core client for 30 sec - exiting 21:24:37 (18912): No heartbeat from core client for 30 sec - exiting 21:24:38 (18912): No heartbeat from core client for 30 sec - exiting 21:24:39 (18912): No heartbeat from core client for 30 sec - exiting 21:24:40 (18912): No heartbeat from core client for 30 sec - exiting 21:24:41 (18912): No heartbeat from core client for 30 sec - exiting 21:24:42 (18912): No heartbeat from core client for 30 sec - exiting 21:24:43 (18912): No heartbeat from core client for 30 sec - exiting 21:24:44 (18912): No heartbeat from core client for 30 sec - exiting 21:24:45 (18912): No heartbeat from core client for 30 sec - exiting 21:24:46 (18912): No heartbeat from core client for 30 sec - exiting 21:24:47 (18912): No heartbeat from core client for 30 sec - exiting 21:24:48 (18912): No heartbeat from core client for 30 sec - exiting 21:24:49 (18912): No heartbeat from core client for 30 sec - exiting 21:24:50 (18912): No heartbeat from core client for 30 sec - exiting 21:24:51 (18912): No heartbeat from core client for 30 sec - exiting 21:24:52 (18912): No heartbeat from core client for 30 sec - exiting 21:24:53 (18912): No heartbeat from core client for 30 sec - exiting 21:24:54 (18912): No heartbeat from core client for 30 sec - exiting 21:24:55 (18912): No heartbeat from core client for 30 sec - exiting 21:24:56 (18912): No heartbeat from core client for 30 sec - exiting 21:24:57 (18912): No heartbeat from core client for 30 sec - exiting 21:24:58 (18912): No heartbeat from core client for 30 sec - exiting 21:28:20 (18922): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:31:50 (18930): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:34:58 (18941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:38:23 (18953): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:40:16 (18969): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:47 (18978): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:18 (18984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:49 (18994): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:50:51 (19001): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:52:06 (19008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:54:38 (19020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:42 (19027): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:01:16 (19042): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:50 (19050): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:15 (19061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:09:58 (19079): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:02:20 (19084): No heartbeat from core client for 30 sec - exiting 23:02:21 (19084): No heartbeat from core client for 30 sec - exiting 23:02:22 (19084): No heartbeat from core client for 30 sec - exiting 23:02:23 (19084): No heartbeat from core client for 30 sec - exiting 23:02:24 (19084): No heartbeat from core client for 30 sec - exiting 23:02:25 (19084): No heartbeat from core client for 30 sec - exiting 23:02:26 (19084): No heartbeat from core client for 30 sec - exiting 23:02:27 (19084): No heartbeat from core client for 30 sec - exiting 23:02:28 (19084): No heartbeat from core client for 30 sec - exiting 23:02:29 (19084): No heartbeat from core client for 30 sec - exiting 23:02:30 (19084): No heartbeat from core client for 30 sec - exiting 23:02:31 (19084): No heartbeat from core client for 30 sec - exiting 23:02:32 (19084): No heartbeat from core client for 30 sec - exiting 23:02:33 (19084): No heartbeat from core client for 30 sec - exiting 23:02:34 (19084): No heartbeat from core client for 30 sec - exiting 23:02:35 (19084): No heartbeat from core client for 30 sec - exiting 23:02:36 (19084): No heartbeat from core client for 30 sec - exiting 23:02:37 (19084): No heartbeat from core client for 30 sec - exiting 23:02:38 (19084): No heartbeat from core client for 30 sec - exiting 23:02:39 (19084): No heartbeat from core client for 30 sec - exiting 23:02:40 (19084): No heartbeat from core client for 30 sec - exiting 23:02:41 (19084): No heartbeat from core client for 30 sec - exiting 23:02:42 (19084): No heartbeat from core client for 30 sec - exiting 23:02:43 (19084): No heartbeat from core client for 30 sec - exiting 23:02:44 (19084): No heartbeat from core client for 30 sec - exiting 23:02:45 (19084): No heartbeat from core client for 30 sec - exiting 23:02:46 (19084): No heartbeat from core client for 30 sec - exiting 23:02:47 (19084): No heartbeat from core client for 30 sec - exiting 23:02:48 (19084): No heartbeat from core client for 30 sec - exiting 23:02:49 (19084): No heartbeat from core client for 30 sec - exiting 23:02:50 (19084): No heartbeat from core client for 30 sec - exiting 23:02:51 (19084): No heartbeat from core client for 30 sec - exiting 23:02:52 (19084): No heartbeat from core client for 30 sec - exiting 23:02:53 (19084): No heartbeat from core client for 30 sec - exiting 23:02:54 (19084): No heartbeat from core client for 30 sec - exiting 23:02:55 (19084): No heartbeat from core client for 30 sec - exiting 23:02:56 (19084): No heartbeat from core client for 30 sec - exiting 23:02:57 (19084): No heartbeat from core client for 30 sec - exiting 23:02:58 (19084): No heartbeat from core client for 30 sec - exiting 23:02:59 (19084): No heartbeat from core client for 30 sec - exiting 23:03:00 (19084): No heartbeat from core client for 30 sec - exiting 23:03:01 (19084): No heartbeat from core client for 30 sec - exiting 23:03:02 (19084): No heartbeat from core client for 30 sec - exiting 23:03:03 (19084): No heartbeat from core client for 30 sec - exiting 23:03:04 (19084): No heartbeat from core client for 30 sec - exiting 23:03:05 (19084): No heartbeat from core client for 30 sec - exiting 23:03:06 (19084): No heartbeat from core client for 30 sec - exiting 23:03:07 (19084): No heartbeat from core client for 30 sec - exiting 23:03:08 (19084): No heartbeat from core client for 30 sec - exiting 23:03:09 (19084): No heartbeat from core client for 30 sec - exiting 23:03:10 (19084): No heartbeat from core client for 30 sec - exiting 23:03:11 (19084): No heartbeat from core client for 30 sec - exiting 23:03:12 (19084): No heartbeat from core client for 30 sec - exiting 23:03:13 (19084): No heartbeat from core client for 30 sec - exiting 23:03:14 (19084): No heartbeat from core client for 30 sec - exiting 23:03:15 (19084): No heartbeat from core client for 30 sec - exiting 23:03:16 (19084): No heartbeat from core client for 30 sec - exiting 23:03:17 (19084): No heartbeat from core client for 30 sec - exiting 23:03:18 (19084): No heartbeat from core client for 30 sec - exiting 23:03:19 (19084): No heartbeat from core client for 30 sec - exiting 23:03:20 (19084): No heartbeat from core client for 30 sec - exiting 23:03:21 (19084): No heartbeat from core client for 30 sec - exiting 23:03:22 (19084): No heartbeat from core client for 30 sec - exiting 23:03:23 (19084): No heartbeat from core client for 30 sec - exiting 23:03:24 (19084): No heartbeat from core client for 30 sec - exiting 23:03:25 (19084): No heartbeat from core client for 30 sec - exiting 23:03:26 (19084): No heartbeat from core client for 30 sec - exiting 23:03:27 (19084): No heartbeat from core client for 30 sec - exiting 23:03:28 (19084): No heartbeat from core client for 30 sec - exiting 23:03:29 (19084): No heartbeat from core client for 30 sec - exiting 23:03:30 (19084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:37 (19207): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:19:47 (19271): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:23:22 (19283): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:21 (19288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:28:09 (19298): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:29:55 (19304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:33:20 (19314): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:37:10 (19328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:39:48 (19343): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:43:03 (19353): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:45:45 (19359): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:48:10 (19368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:48:19 (19368): No heartbeat from core client for 30 sec - exiting 23:51:37 (19377): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:21 (19387): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:59:16 (19396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:09:53 (19429): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:27 (19435): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:14:20 (19577): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:17:32 (19586): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:20:35 (19619): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:00 (19627): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:26 (19633): No heartbeat from core client for 30 sec - exiting 01:24:27 (19633): No heartbeat from core client for 30 sec - exiting 01:24:28 (19633): No heartbeat from core client for 30 sec - exiting 01:24:29 (19633): No heartbeat from core client for 30 sec - exiting 01:24:30 (19633): No heartbeat from core client for 30 sec - exiting 01:24:31 (19633): No heartbeat from core client for 30 sec - exiting 01:24:32 (19633): No heartbeat from core client for 30 sec - exiting 01:24:33 (19633): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:43:36 (19640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:44:16 (19679): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:47:39 (19689): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:50:29 (19696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:52:39 (19706): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:53:43 (19711): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:56:51 (19721): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:28:35 (19727): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:28:36 (19727): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xd49400] [0xd49422] /lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x8e9651] /lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x8eca82] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x8d5bd6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20973, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xdac400] [0xdac422] /lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x9b9651] /lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x9bca82] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x9a5bd6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20973, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0x8ef400] [0x8ef422] /lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x5fd651] /lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x600a82] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x5e9bd6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20973, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xea6400] [0xea6422] /lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x6e6651] /lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x6e9a82] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x6d2bd6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20973, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xfe6400] [0xfe6422] /lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x63d651] /lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x640a82] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x629bd6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20973, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0x972400] [0x972422] /lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x13e651] /lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x141a82] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x12abd6] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20973, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 May 2011 00:09:29	845875	12829737	hadcm3n_p5nd_1900_40_007224905_0	51,840	190,246	3.6699
29 Apr 2011 04:16:12	845875	12829737	hadcm3n_p5nd_1900_40_007224905_0	25,920	95,114	3.6695