Task 15632130

Name	hadcm3n_3mhm_1980_40_008318155_0
Workunit	8469290
Created	24 Feb 2013, 4:15:50 UTC
Sent	24 Feb 2013, 4:16:01 UTC
Report deadline	26 May 2013, 11:43:12 UTC
Received	14 Apr 2013, 15:19:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1170039
Run time	12 days 8 hours 58 min 38 sec
CPU time	12 days 8 hours 17 min 6 sec
Validate state	Invalid
Credit	8,398.08
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.29</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:07:41 (6372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:17 (6985): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:11:27 (7047): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:20 (7075): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:19:56 (7163): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:44 (7245): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:22:06 (7263): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:22:09 (7263): No heartbeat from core client for 30 sec - exiting 13:26:25 (7300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:27:13 (7383): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:29:38 (7408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:32:26 (7447): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:34:12 (7528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:38:49 (7565): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:39:43 (7648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:40:28 (7669): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:42:41 (7703): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:43:46 (7750): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:43:47 (7750): No heartbeat from core client for 30 sec - exiting 13:43:48 (7750): No heartbeat from core client for 30 sec - exiting 13:43:49 (7750): No heartbeat from core client for 30 sec - exiting 13:43:50 (7750): No heartbeat from core client for 30 sec - exiting 13:43:51 (7750): No heartbeat from core client for 30 sec - exiting 13:43:52 (7750): No heartbeat from core client for 30 sec - exiting 13:46:40 (7771): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:47:36 (7830): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:48:25 (7852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:52 (7874): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:00:20 (7920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:00:21 (7920): No heartbeat from core client for 30 sec - exiting 14:00:22 (7920): No heartbeat from core client for 30 sec - exiting 14:00:23 (7920): No heartbeat from core client for 30 sec - exiting 14:00:24 (7920): No heartbeat from core client for 30 sec - exiting 14:00:25 (7920): No heartbeat from core client for 30 sec - exiting 14:00:26 (7920): No heartbeat from core client for 30 sec - exiting 14:00:27 (7920): No heartbeat from core client for 30 sec - exiting 14:00:28 (7920): No heartbeat from core client for 30 sec - exiting 14:00:29 (7920): No heartbeat from core client for 30 sec - exiting 14:00:30 (7920): No heartbeat from core client for 30 sec - exiting 14:00:31 (7920): No heartbeat from core client for 30 sec - exiting 14:00:32 (7920): No heartbeat from core client for 30 sec - exiting 14:00:33 (7920): No heartbeat from core client for 30 sec - exiting 14:00:34 (7920): No heartbeat from core client for 30 sec - exiting 14:00:35 (7920): No heartbeat from core client for 30 sec - exiting 14:00:36 (7920): No heartbeat from core client for 30 sec - exiting 14:00:37 (7920): No heartbeat from core client for 30 sec - exiting 14:00:38 (7920): No heartbeat from core client for 30 sec - exiting 14:00:39 (7920): No heartbeat from core client for 30 sec - exiting 14:00:40 (7920): No heartbeat from core client for 30 sec - exiting 14:00:41 (7920): No heartbeat from core client for 30 sec - exiting 14:00:42 (7920): No heartbeat from core client for 30 sec - exiting 14:00:43 (7920): No heartbeat from core client for 30 sec - exiting 14:00:44 (7920): No heartbeat from core client for 30 sec - exiting 14:00:45 (7920): No heartbeat from core client for 30 sec - exiting 14:00:46 (7920): No heartbeat from core client for 30 sec - exiting 14:00:47 (7920): No heartbeat from core client for 30 sec - exiting 14:00:48 (7920): No heartbeat from core client for 30 sec - exiting 14:02:59 (8115): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:10:23 (8162): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:12:01 (8310): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:12:02 (8310): No heartbeat from core client for 30 sec - exiting 14:12:03 (8310): No heartbeat from core client for 30 sec - exiting 14:12:04 (8310): No heartbeat from core client for 30 sec - exiting 14:12:05 (8310): No heartbeat from core client for 30 sec - exiting 14:12:06 (8310): No heartbeat from core client for 30 sec - exiting 14:12:07 (8310): No heartbeat from core client for 30 sec - exiting 14:12:08 (8310): No heartbeat from core client for 30 sec - exiting 14:12:09 (8310): No heartbeat from core client for 30 sec - exiting 14:12:10 (8310): No heartbeat from core client for 30 sec - exiting 14:12:11 (8310): No heartbeat from core client for 30 sec - exiting 14:12:12 (8310): No heartbeat from core client for 30 sec - exiting 14:12:13 (8310): No heartbeat from core client for 30 sec - exiting 14:12:14 (8310): No heartbeat from core client for 30 sec - exiting 14:12:15 (8310): No heartbeat from core client for 30 sec - exiting 14:12:16 (8310): No heartbeat from core client for 30 sec - exiting 14:12:17 (8310): No heartbeat from core client for 30 sec - exiting 14:12:18 (8310): No heartbeat from core client for 30 sec - exiting 14:12:19 (8310): No heartbeat from core client for 30 sec - exiting 14:12:20 (8310): No heartbeat from core client for 30 sec - exiting 14:12:21 (8310): No heartbeat from core client for 30 sec - exiting 14:12:22 (8310): No heartbeat from core client for 30 sec - exiting 14:12:23 (8310): No heartbeat from core client for 30 sec - exiting 14:13:35 (8344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:58 (8373): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:20:59 (8373): No heartbeat from core client for 30 sec - exiting 14:21:00 (8373): No heartbeat from core client for 30 sec - exiting 14:21:01 (8373): No heartbeat from core client for 30 sec - exiting 14:21:02 (8373): No heartbeat from core client for 30 sec - exiting 14:21:03 (8373): No heartbeat from core client for 30 sec - exiting 14:21:04 (8373): No heartbeat from core client for 30 sec - exiting 14:21:05 (8373): No heartbeat from core client for 30 sec - exiting 14:21:06 (8373): No heartbeat from core client for 30 sec - exiting 14:21:07 (8373): No heartbeat from core client for 30 sec - exiting 14:21:08 (8373): No heartbeat from core client for 30 sec - exiting 14:21:09 (8373): No heartbeat from core client for 30 sec - exiting 14:21:10 (8373): No heartbeat from core client for 30 sec - exiting 14:21:11 (8373): No heartbeat from core client for 30 sec - exiting 14:21:12 (8373): No heartbeat from core client for 30 sec - exiting 14:21:13 (8373): No heartbeat from core client for 30 sec - exiting 14:21:14 (8373): No heartbeat from core client for 30 sec - exiting 14:21:15 (8373): No heartbeat from core client for 30 sec - exiting 14:21:16 (8373): No heartbeat from core client for 30 sec - exiting 14:21:17 (8373): No heartbeat from core client for 30 sec - exiting 14:21:18 (8373): No heartbeat from core client for 30 sec - exiting 14:21:19 (8373): No heartbeat from core client for 30 sec - exiting 14:21:20 (8373): No heartbeat from core client for 30 sec - exiting 14:21:21 (8373): No heartbeat from core client for 30 sec - exiting 14:21:22 (8373): No heartbeat from core client for 30 sec - exiting 14:21:23 (8373): No heartbeat from core client for 30 sec - exiting 14:21:24 (8373): No heartbeat from core client for 30 sec - exiting 14:21:25 (8373): No heartbeat from core client for 30 sec - exiting 14:21:26 (8373): No heartbeat from core client for 30 sec - exiting 14:21:27 (8373): No heartbeat from core client for 30 sec - exiting 14:21:28 (8373): No heartbeat from core client for 30 sec - exiting 14:21:29 (8373): No heartbeat from core client for 30 sec - exiting 14:21:30 (8373): No heartbeat from core client for 30 sec - exiting 14:22:05 (8523): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:23:30 (8544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:23:31 (8544): No heartbeat from core client for 30 sec - exiting 14:23:32 (8544): No heartbeat from core client for 30 sec - exiting 14:23:33 (8544): No heartbeat from core client for 30 sec - exiting 14:23:34 (8544): No heartbeat from core client for 30 sec - exiting 14:23:35 (8544): No heartbeat from core client for 30 sec - exiting 14:23:36 (8544): No heartbeat from core client for 30 sec - exiting 14:23:37 (8544): No heartbeat from core client for 30 sec - exiting 14:23:38 (8544): No heartbeat from core client for 30 sec - exiting 14:23:39 (8544): No heartbeat from core client for 30 sec - exiting 14:23:40 (8544): No heartbeat from core client for 30 sec - exiting 14:23:41 (8544): No heartbeat from core client for 30 sec - exiting 14:23:42 (8544): No heartbeat from core client for 30 sec - exiting 14:23:43 (8544): No heartbeat from core client for 30 sec - exiting 14:23:44 (8544): No heartbeat from core client for 30 sec - exiting 14:23:45 (8544): No heartbeat from core client for 30 sec - exiting 14:23:46 (8544): No heartbeat from core client for 30 sec - exiting 14:23:47 (8544): No heartbeat from core client for 30 sec - exiting 14:23:48 (8544): No heartbeat from core client for 30 sec - exiting 14:23:49 (8544): No heartbeat from core client for 30 sec - exiting 14:23:50 (8544): No heartbeat from core client for 30 sec - exiting 14:23:51 (8544): No heartbeat from core client for 30 sec - exiting 14:23:52 (8544): No heartbeat from core client for 30 sec - exiting 14:23:53 (8544): No heartbeat from core client for 30 sec - exiting 14:23:54 (8544): No heartbeat from core client for 30 sec - exiting 14:23:55 (8544): No heartbeat from core client for 30 sec - exiting 14:23:56 (8544): No heartbeat from core client for 30 sec - exiting 14:23:57 (8544): No heartbeat from core client for 30 sec - exiting 14:23:58 (8544): No heartbeat from core client for 30 sec - exiting 14:23:59 (8544): No heartbeat from core client for 30 sec - exiting 14:26:01 (8583): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:26:02 (8583): No heartbeat from core client for 30 sec - exiting 14:26:03 (8583): No heartbeat from core client for 30 sec - exiting 14:26:04 (8583): No heartbeat from core client for 30 sec - exiting 14:28:06 (8625): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:55 (8670): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:56 (8670): No heartbeat from core client for 30 sec - exiting 14:29:57 (8670): No heartbeat from core client for 30 sec - exiting 14:29:58 (8670): No heartbeat from core client for 30 sec - exiting 14:29:59 (8670): No heartbeat from core client for 30 sec - exiting 14:30:00 (8670): No heartbeat from core client for 30 sec - exiting 14:30:01 (8670): No heartbeat from core client for 30 sec - exiting 14:30:02 (8670): No heartbeat from core client for 30 sec - exiting 14:30:03 (8670): No heartbeat from core client for 30 sec - exiting 14:30:04 (8670): No heartbeat from core client for 30 sec - exiting 14:30:05 (8670): No heartbeat from core client for 30 sec - exiting 14:30:06 (8670): No heartbeat from core client for 30 sec - exiting 14:30:07 (8670): No heartbeat from core client for 30 sec - exiting 14:30:08 (8670): No heartbeat from core client for 30 sec - exiting 14:30:09 (8670): No heartbeat from core client for 30 sec - exiting 14:30:10 (8670): No heartbeat from core client for 30 sec - exiting 14:30:11 (8670): No heartbeat from core client for 30 sec - exiting 14:30:12 (8670): No heartbeat from core client for 30 sec - exiting 14:30:13 (8670): No heartbeat from core client for 30 sec - exiting 14:30:14 (8670): No heartbeat from core client for 30 sec - exiting 14:30:15 (8670): No heartbeat from core client for 30 sec - exiting 14:30:16 (8670): No heartbeat from core client for 30 sec - exiting 14:30:17 (8670): No heartbeat from core client for 30 sec - exiting 14:30:18 (8670): No heartbeat from core client for 30 sec - exiting 14:30:19 (8670): No heartbeat from core client for 30 sec - exiting 14:30:20 (8670): No heartbeat from core client for 30 sec - exiting 14:30:21 (8670): No heartbeat from core client for 30 sec - exiting 14:30:22 (8670): No heartbeat from core client for 30 sec - exiting 14:30:23 (8670): No heartbeat from core client for 30 sec - exiting 14:30:24 (8670): No heartbeat from core client for 30 sec - exiting 14:30:25 (8670): No heartbeat from core client for 30 sec - exiting 14:30:26 (8670): No heartbeat from core client for 30 sec - exiting 14:30:27 (8670): No heartbeat from core client for 30 sec - exiting 14:32:20 (8733): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:32:21 (8733): No heartbeat from core client for 30 sec - exiting 14:32:22 (8733): No heartbeat from core client for 30 sec - exiting 14:32:23 (8733): No heartbeat from core client for 30 sec - exiting 14:32:24 (8733): No heartbeat from core client for 30 sec - exiting 14:32:25 (8733): No heartbeat from core client for 30 sec - exiting 14:32:26 (8733): No heartbeat from core client for 30 sec - exiting 14:32:27 (8733): No heartbeat from core client for 30 sec - exiting 14:32:28 (8733): No heartbeat from core client for 30 sec - exiting 14:32:29 (8733): No heartbeat from core client for 30 sec - exiting 14:32:30 (8733): No heartbeat from core client for 30 sec - exiting 14:32:31 (8733): No heartbeat from core client for 30 sec - exiting 14:32:32 (8733): No heartbeat from core client for 30 sec - exiting 14:32:33 (8733): No heartbeat from core client for 30 sec - exiting 14:32:34 (8733): No heartbeat from core client for 30 sec - exiting 14:32:35 (8733): No heartbeat from core client for 30 sec - exiting 14:32:36 (8733): No heartbeat from core client for 30 sec - exiting 14:32:37 (8733): No heartbeat from core client for 30 sec - exiting 14:32:38 (8733): No heartbeat from core client for 30 sec - exiting 14:32:39 (8733): No heartbeat from core client for 30 sec - exiting 14:32:40 (8733): No heartbeat from core client for 30 sec - exiting 14:32:41 (8733): No heartbeat from core client for 30 sec - exiting 14:32:42 (8733): No heartbeat from core client for 30 sec - exiting 14:32:43 (8733): No heartbeat from core client for 30 sec - exiting 14:32:44 (8733): No heartbeat from core client for 30 sec - exiting 14:32:45 (8733): No heartbeat from core client for 30 sec - exiting 14:32:46 (8733): No heartbeat from core client for 30 sec - exiting 14:32:47 (8733): No heartbeat from core client for 30 sec - exiting 14:32:48 (8733): No heartbeat from core client for 30 sec - exiting 14:32:49 (8733): No heartbeat from core client for 30 sec - exiting 14:41:30 (8771): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:43:11 (8952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:46:57 (8986): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 15:00:42 (9054): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:00:43 (9054): No heartbeat from core client for 30 sec - exiting 15:00:44 (9054): No heartbeat from core client for 30 sec - exiting 15:02:21 (9305): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:52 (9342): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:32:41 (9390): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:35:15 (9887): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:26 (9939): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:38:42 (9986): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:41:45 (10007): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:44:58 (10076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:47:12 (10140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:47:13 (10140): No heartbeat from core client for 30 sec - exiting 15:47:14 (10140): No heartbeat from core client for 30 sec - exiting 15:47:15 (10140): No heartbeat from core client for 30 sec - exiting 15:47:16 (10140): No heartbeat from core client for 30 sec - exiting 15:47:17 (10140): No heartbeat from core client for 30 sec - exiting 15:47:18 (10140): No heartbeat from core client for 30 sec - exiting 15:47:19 (10140): No heartbeat from core client for 30 sec - exiting 15:47:20 (10140): No heartbeat from core client for 30 sec - exiting 15:47:21 (10140): No heartbeat from core client for 30 sec - exiting 15:47:22 (10140): No heartbeat from core client for 30 sec - exiting 15:47:23 (10140): No heartbeat from core client for 30 sec - exiting 15:47:24 (10140): No heartbeat from core client for 30 sec - exiting 15:47:25 (10140): No heartbeat from core client for 30 sec - exiting 15:47:26 (10140): No heartbeat from core client for 30 sec - exiting 15:47:27 (10140): No heartbeat from core client for 30 sec - exiting 15:47:28 (10140): No heartbeat from core client for 30 sec - exiting 15:47:29 (10140): No heartbeat from core client for 30 sec - exiting 15:47:30 (10140): No heartbeat from core client for 30 sec - exiting 15:47:31 (10140): No heartbeat from core client for 30 sec - exiting 15:47:32 (10140): No heartbeat from core client for 30 sec - exiting 15:47:33 (10140): No heartbeat from core client for 30 sec - exiting 15:47:34 (10140): No heartbeat from core client for 30 sec - exiting 15:47:35 (10140): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7705400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf7705425] /lib32/libc.so.6(gsignal+0x49)[0xf75336e9] /lib32/libc.so.6(abort+0x18b)[0xf7534f6b] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf751f5a6] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10187, iMonCtr=1 Model crash detected, will try to restart... 15:48:09 (10187): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:22 (10187): No heartbeat from core client for 30 sec - exiting 15:48:23 (10187): No heartbeat from core client for 30 sec - exiting 15:48:24 (10187): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf777c400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf777c425] /lib32/libc.so.6(gsignal+0x49)[0xf75aa6e9] /lib32/libc.so.6(abort+0x18b)[0xf75abf6b] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75965a6] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10210, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77a4400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf77a4425] /lib32/libc.so.6(gsignal+0x49)[0xf75d26e9] /lib32/libc.so.6(abort+0x18b)[0xf75d3f6b] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75be5a6] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10210, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77d7400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf77d7425] /lib32/libc.so.6(gsignal+0x49)[0xf76056e9] /lib32/libc.so.6(abort+0x18b)[0xf7606f6b] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75f15a6] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10210, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77a3400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf77a3425] /lib32/libc.so.6(gsignal+0x49)[0xf75d16e9] /lib32/libc.so.6(abort+0x18b)[0xf75d2f6b] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75bd5a6] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10210, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7757400] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf7757425] /lib32/libc.so.6(gsignal+0x49)[0xf75856e9] /lib32/libc.so.6(abort+0x18b)[0xf7586f6b] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75715a6] /var/lib/boinc/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10210, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Apr 2013 18:40:47	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	699,840	1,042,657	1.4899
12 Apr 2013 03:22:25	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	673,920	1,002,696	1.4879
10 Apr 2013 19:53:14	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	648,000	964,012	1.4877
09 Apr 2013 05:43:54	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	622,080	923,122	1.4839
08 Apr 2013 04:45:36	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	596,160	884,076	1.4830
06 Apr 2013 08:13:27	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	570,240	846,012	1.4836
04 Apr 2013 15:20:39	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	544,320	806,085	1.4809
03 Apr 2013 13:22:24	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	518,400	766,318	1.4782
02 Apr 2013 20:19:21	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	492,480	726,371	1.4749
30 Mar 2013 05:13:17	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	466,560	686,589	1.4716
27 Mar 2013 22:09:28	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	440,640	646,525	1.4672
25 Mar 2013 02:25:40	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	414,720	606,506	1.4624
24 Mar 2013 07:41:49	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	388,800	566,738	1.4577
23 Mar 2013 20:16:15	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	362,880	525,949	1.4494
22 Mar 2013 08:13:46	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	336,960	486,562	1.4440
20 Mar 2013 01:35:18	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	311,040	448,571	1.4422
17 Mar 2013 15:00:30	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	285,120	411,350	1.4427
16 Mar 2013 11:39:13	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	259,200	374,745	1.4458
13 Mar 2013 17:16:09	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	233,280	337,552	1.4470
08 Mar 2013 11:32:23	1170039	15632130	hadcm3n_3mhm_1980_40_008318155_0	207,360	300,622	1.4498