Task 12926942

Name	hadcm3n_o6yp_1940_40_007266752_1
Workunit	7464992
Created	2 Jun 2011, 23:27:47 UTC
Sent	2 Jun 2011, 23:27:51 UTC
Report deadline	2 Sep 2011, 6:55:02 UTC
Received	23 Jul 2011, 1:13:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1050772
Run time	17 days 23 hours 25 min 8 sec
CPU time	7 days 7 hours 48 min 20 sec
Validate state	Invalid
Credit	311.04
Device peak FLOPS	0.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xb7858400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf3)[0x411de513] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... 23:37:52 (1889): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:48:53 (2516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:23 (7628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:53:29 (8788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 04:58:40 (9061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:43:06 (11019): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:46:10 (11406): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:48:37 (11418): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:36:25 (11668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:36:26 (11668): No heartbeat from core client for 30 sec - exiting 19:36:27 (11668): No heartbeat from core client for 30 sec - exiting 19:36:28 (11668): No heartbeat from core client for 30 sec - exiting 19:36:29 (11668): No heartbeat from core client for 30 sec - exiting 19:36:30 (11668): No heartbeat from core client for 30 sec - exiting 19:36:31 (11668): No heartbeat from core client for 30 sec - exiting 19:36:32 (11668): No heartbeat from core client for 30 sec - exiting 20:14:07 (11928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:45:27 (11962): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:41:44 (12054): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:20:07 (16625): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:02:26 (16658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:02:27 (16658): No heartbeat from core client for 30 sec - exiting 05:02:28 (16658): No heartbeat from core client for 30 sec - exiting 05:02:29 (16658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:43:33 (1941): No heartbeat from core client for 30 sec - exiting 17:43:34 (1941): No heartbeat from core client for 30 sec - exiting 17:43:35 (1941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:36 (1941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:12:28 (16040): No heartbeat from core client for 30 sec - exiting 11:12:32 (16040): No heartbeat from core client for 30 sec - exiting 11:12:33 (16040): No heartbeat from core client for 30 sec - exiting 11:12:34 (16040): No heartbeat from core client for 30 sec - exiting 11:12:35 (16040): No heartbeat from core client for 30 sec - exiting 11:12:36 (16040): No heartbeat from core client for 30 sec - exiting 11:12:37 (16040): No heartbeat from core client for 30 sec - exiting 11:12:38 (16040): No heartbeat from core client for 30 sec - exiting 11:12:39 (16040): No heartbeat from core client for 30 sec - exiting 11:12:40 (16040): No heartbeat from core client for 30 sec - exiting 11:12:41 (16040): No heartbeat from core client for 30 sec - exiting 11:12:42 (16040): No heartbeat from core client for 30 sec - exiting 11:12:43 (16040): No heartbeat from core client for 30 sec - exiting 11:12:53 (16040): No heartbeat from core client for 30 sec - exiting 11:12:54 (16040): No heartbeat from core client for 30 sec - exiting 11:12:56 (16040): No heartbeat from core client for 30 sec - exiting 11:12:57 (16040): No heartbeat from core client for 30 sec - exiting 11:12:58 (16040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:14:07 (16175): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:49:44 (19417): No heartbeat from core client for 30 sec - exiting 03:49:45 (19417): No heartbeat from core client for 30 sec - exiting 03:49:46 (19417): No heartbeat from core client for 30 sec - exiting 03:49:47 (19417): No heartbeat from core client for 30 sec - exiting 03:49:48 (19417): No heartbeat from core client for 30 sec - exiting 03:49:49 (19417): No heartbeat from core client for 30 sec - exiting 03:49:50 (19417): No heartbeat from core client for 30 sec - exiting 03:49:51 (19417): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 05:18:13 (19475): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:29:03 (19818): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:13:52 (20234): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:14:57 (20263): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:15:28 (20263): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:19:12 (26367): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:21:18 (26401): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:44:29 (26408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:15:52 (28076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:37:08 (28939): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:01:06 (29651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:07 (29651): No heartbeat from core client for 30 sec - exiting 18:01:08 (29651): No heartbeat from core client for 30 sec - exiting 18:01:09 (29651): No heartbeat from core client for 30 sec - exiting 18:01:10 (29651): No heartbeat from core client for 30 sec - exiting 18:01:11 (29651): No heartbeat from core client for 30 sec - exiting 18:01:12 (29651): No heartbeat from core client for 30 sec - exiting 18:01:13 (29651): No heartbeat from core client for 30 sec - exiting 18:01:14 (29651): No heartbeat from core client for 30 sec - exiting 18:01:15 (29651): No heartbeat from core client for 30 sec - exiting 18:01:16 (29651): No heartbeat from core client for 30 sec - exiting 18:01:17 (29651): No heartbeat from core client for 30 sec - exiting 18:01:18 (29651): No heartbeat from core client for 30 sec - exiting 18:01:19 (29651): No heartbeat from core client for 30 sec - exiting 18:01:20 (29651): No heartbeat from core client for 30 sec - exiting 18:01:21 (29651): No heartbeat from core client for 30 sec - exiting 18:01:22 (29651): No heartbeat from core client for 30 sec - exiting 18:01:23 (29651): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:37:44 (30224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:00:44 (30557): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:45:46 (31364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:35:43 (3718): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:47 (3825): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:48 (3825): No heartbeat from core client for 30 sec - exiting 22:00:49 (3825): No heartbeat from core client for 30 sec - exiting 22:00:50 (3825): No heartbeat from core client for 30 sec - exiting 22:00:51 (3825): No heartbeat from core client for 30 sec - exiting 22:00:52 (3825): No heartbeat from core client for 30 sec - exiting 22:00:53 (3825): No heartbeat from core client for 30 sec - exiting 22:00:54 (3825): No heartbeat from core client for 30 sec - exiting 22:00:55 (3825): No heartbeat from core client for 30 sec - exiting 22:00:56 (3825): No heartbeat from core client for 30 sec - exiting 22:00:57 (3825): No heartbeat from core client for 30 sec - exiting 22:00:58 (3825): No heartbeat from core client for 30 sec - exiting 22:00:59 (3825): No heartbeat from core client for 30 sec - exiting 22:01:00 (3825): No heartbeat from core client for 30 sec - exiting 22:01:01 (3825): No heartbeat from core client for 30 sec - exiting 22:01:02 (3825): No heartbeat from core client for 30 sec - exiting 22:01:03 (3825): No heartbeat from core client for 30 sec - exiting 22:01:04 (3825): No heartbeat from core client for 30 sec - exiting 22:01:05 (3825): No heartbeat from core client for 30 sec - exiting 22:01:06 (3825): No heartbeat from core client for 30 sec - exiting 22:01:07 (3825): No heartbeat from core client for 30 sec - exiting 22:01:08 (3825): No heartbeat from core client for 30 sec - exiting 22:01:09 (3825): No heartbeat from core client for 30 sec - exiting 22:01:10 (3825): No heartbeat from core client for 30 sec - exiting 22:01:11 (3825): No heartbeat from core client for 30 sec - exiting 22:01:12 (3825): No heartbeat from core client for 30 sec - exiting 22:01:13 (3825): No heartbeat from core client for 30 sec - exiting 22:01:14 (3825): No heartbeat from core client for 30 sec - exiting 22:01:15 (3825): No heartbeat from core client for 30 sec - exiting 22:01:16 (3825): No heartbeat from core client for 30 sec - exiting 00:22:56 (4045): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:23:48 (4138): No heartbeat from core client for 30 sec - exiting 00:24:12 (4138): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:24:38 (4144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:15:34 (14952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:46:15 (15269): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:46:16 (15269): No heartbeat from core client for 30 sec - exiting 18:46:17 (15269): No heartbeat from core client for 30 sec - exiting 18:46:18 (15269): No heartbeat from core client for 30 sec - exiting 18:46:19 (15269): No heartbeat from core client for 30 sec - exiting 18:46:20 (15269): No heartbeat from core client for 30 sec - exiting 18:46:21 (15269): No heartbeat from core client for 30 sec - exiting 21:48:14 (15505): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:48 (15629): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:35:30 (16420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:01 (16583): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:03 (16932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:04 (16932): No heartbeat from core client for 30 sec - exiting 11:26:05 (16932): No heartbeat from core client for 30 sec - exiting 11:26:06 (16932): No heartbeat from core client for 30 sec - exiting Signal 1 received, exiting... Called boinc_finish 22:41:36 (10129): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:43:11 (10289): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:43:12 (10289): No heartbeat from core client for 30 sec - exiting 05:43:13 (10289): No heartbeat from core client for 30 sec - exiting 05:43:14 (10289): No heartbeat from core client for 30 sec - exiting 05:43:15 (10289): No heartbeat from core client for 30 sec - exiting 05:43:16 (10289): No heartbeat from core client for 30 sec - exiting 05:43:17 (10289): No heartbeat from core client for 30 sec - exiting 05:43:18 (10289): No heartbeat from core client for 30 sec - exiting 05:43:19 (10289): No heartbeat from core client for 30 sec - exiting 05:43:20 (10289): No heartbeat from core client for 30 sec - exiting 05:43:21 (10289): No heartbeat from core client for 30 sec - exiting 06:13:14 (10643): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:13:15 (10643): No heartbeat from core client for 30 sec - exiting 06:13:16 (10643): No heartbeat from core client for 30 sec - exiting 06:13:17 (10643): No heartbeat from core client for 30 sec - exiting 06:13:18 (10643): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 02:38:01 (1905): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:59:15 (2315): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:58 (4805): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:59 (4805): No heartbeat from core client for 30 sec - exiting 21:19:00 (4805): No heartbeat from core client for 30 sec - exiting 21:19:01 (4805): No heartbeat from core client for 30 sec - exiting 21:19:02 (4805): No heartbeat from core client for 30 sec - exiting 21:19:03 (4805): No heartbeat from core client for 30 sec - exiting 21:19:04 (4805): No heartbeat from core client for 30 sec - exiting 21:19:06 (4805): No heartbeat from core client for 30 sec - exiting 21:19:07 (4805): No heartbeat from core client for 30 sec - exiting 21:19:08 (4805): No heartbeat from core client for 30 sec - exiting 21:19:09 (4805): No heartbeat from core client for 30 sec - exiting 21:19:10 (4805): No heartbeat from core client for 30 sec - exiting 21:19:11 (4805): No heartbeat from core client for 30 sec - exiting 21:19:12 (4805): No heartbeat from core client for 30 sec - exiting 21:19:13 (4805): No heartbeat from core client for 30 sec - exiting 21:19:14 (4805): No heartbeat from core client for 30 sec - exiting 21:19:15 (4805): No heartbeat from core client for 30 sec - exiting 21:19:16 (4805): No heartbeat from core client for 30 sec - exiting 21:19:17 (4805): No heartbeat from core client for 30 sec - exiting 21:19:18 (4805): No heartbeat from core client for 30 sec - exiting 21:19:19 (4805): No heartbeat from core client for 30 sec - exiting 21:19:20 (4805): No heartbeat from core client for 30 sec - exiting 21:19:21 (4805): No heartbeat from core client for 30 sec - exiting 21:19:22 (4805): No heartbeat from core client for 30 sec - exiting 21:19:23 (4805): No heartbeat from core client for 30 sec - exiting 21:19:24 (4805): No heartbeat from core client for 30 sec - exiting 21:19:25 (4805): No heartbeat from core client for 30 sec - exiting 21:19:26 (4805): No heartbeat from core client for 30 sec - exiting 21:19:27 (4805): No heartbeat from core client for 30 sec - exiting 21:19:28 (4805): No heartbeat from core client for 30 sec - exiting 21:19:29 (4805): No heartbeat from core client for 30 sec - exiting 21:19:30 (4805): No heartbeat from core client for 30 sec - exiting 21:19:31 (4805): No heartbeat from core client for 30 sec - exiting 21:19:32 (4805): No heartbeat from core client for 30 sec - exiting 21:19:33 (4805): No heartbeat from core client for 30 sec - exiting 21:19:34 (4805): No heartbeat from core client for 30 sec - exiting 21:19:35 (4805): No heartbeat from core client for 30 sec - exiting 21:19:36 (4805): No heartbeat from core client for 30 sec - exiting 21:19:37 (4805): No heartbeat from core client for 30 sec - exiting 22:48:54 (5008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:55 (5008): No heartbeat from core client for 30 sec - exiting 22:48:56 (5008): No heartbeat from core client for 30 sec - exiting 22:48:57 (5008): No heartbeat from core client for 30 sec - exiting 22:48:58 (5008): No heartbeat from core client for 30 sec - exiting 22:48:59 (5008): No heartbeat from core client for 30 sec - exiting 22:49:00 (5008): No heartbeat from core client for 30 sec - exiting 22:49:01 (5008): No heartbeat from core client for 30 sec - exiting 22:49:02 (5008): No heartbeat from core client for 30 sec - exiting 22:49:03 (5008): No heartbeat from core client for 30 sec - exiting 23:44:27 (5067): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:44:28 (5067): No heartbeat from core client for 30 sec - exiting 23:44:29 (5067): No heartbeat from core client for 30 sec - exiting 23:44:30 (5067): No heartbeat from core client for 30 sec - exiting 23:44:31 (5067): No heartbeat from core client for 30 sec - exiting 23:44:32 (5067): No heartbeat from core client for 30 sec - exiting 23:44:33 (5067): No heartbeat from core client for 30 sec - exiting 23:44:34 (5067): No heartbeat from core client for 30 sec - exiting 23:44:35 (5067): No heartbeat from core client for 30 sec - exiting 03:29:49 (5110): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:26:30 (1941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:28:37 (5559): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:28:39 (5559): No heartbeat from core client for 30 sec - exiting 03:28:40 (5559): No heartbeat from core client for 30 sec - exiting 03:28:41 (5559): No heartbeat from core client for 30 sec - exiting 03:28:42 (5559): No heartbeat from core client for 30 sec - exiting 04:58:53 (5998): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:58 (6052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:29 (6086): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:30 (6086): No heartbeat from core client for 30 sec - exiting 06:00:31 (6086): No heartbeat from core client for 30 sec - exiting 06:00:32 (6086): No heartbeat from core client for 30 sec - exiting 06:00:33 (6086): No heartbeat from core client for 30 sec - exiting 06:00:34 (6086): No heartbeat from core client for 30 sec - exiting 06:00:35 (6086): No heartbeat from core client for 30 sec - exiting 06:00:36 (6086): No heartbeat from core client for 30 sec - exiting 06:00:37 (6086): No heartbeat from core client for 30 sec - exiting 06:00:38 (6086): No heartbeat from core client for 30 sec - exiting 06:00:39 (6086): No heartbeat from core client for 30 sec - exiting 06:00:40 (6086): No heartbeat from core client for 30 sec - exiting 06:00:41 (6086): No heartbeat from core client for 30 sec - exiting 06:00:42 (6086): No heartbeat from core client for 30 sec - exiting 08:34:30 (6106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:41 (6219): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:25 (6251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:31:59 (6336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:03:46 (6433): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:32:21 (6469): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:00 (6521): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:01 (6521): No heartbeat from core client for 30 sec - exiting 16:23:02 (6521): No heartbeat from core client for 30 sec - exiting 16:23:03 (6521): No heartbeat from core client for 30 sec - exiting 16:23:04 (6521): No heartbeat from core client for 30 sec - exiting 16:23:05 (6521): No heartbeat from core client for 30 sec - exiting 16:23:06 (6521): No heartbeat from core client for 30 sec - exiting 16:23:07 (6521): No heartbeat from core client for 30 sec - exiting 16:23:08 (6521): No heartbeat from core client for 30 sec - exiting 16:23:09 (6521): No heartbeat from core client for 30 sec - exiting 16:23:10 (6521): No heartbeat from core client for 30 sec - exiting 16:23:11 (6521): No heartbeat from core client for 30 sec - exiting 16:23:12 (6521): No heartbeat from core client for 30 sec - exiting 16:23:13 (6521): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:26:13 (6564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:12:03 (7838): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:12:04 (7838): No heartbeat from core client for 30 sec - exiting 22:12:05 (7838): No heartbeat from core client for 30 sec - exiting 22:12:06 (7838): No heartbeat from core client for 30 sec - exiting 22:12:07 (7838): No heartbeat from core client for 30 sec - exiting 22:12:08 (7838): No heartbeat from core client for 30 sec - exiting 22:12:09 (7838): No heartbeat from core client for 30 sec - exiting 22:12:10 (7838): No heartbeat from core client for 30 sec - exiting 22:12:11 (7838): No heartbeat from core client for 30 sec - exiting 22:12:12 (7838): No heartbeat from core client for 30 sec - exiting 22:12:13 (7838): No heartbeat from core client for 30 sec - exiting 22:12:14 (7838): No heartbeat from core client for 30 sec - exiting 22:12:15 (7838): No heartbeat from core client for 30 sec - exiting 22:12:17 (7838): No heartbeat from core client for 30 sec - exiting 22:12:18 (7838): No heartbeat from core client for 30 sec - exiting 22:12:19 (7838): No heartbeat from core client for 30 sec - exiting 22:12:20 (7838): No heartbeat from core client for 30 sec - exiting 22:12:21 (7838): No heartbeat from core client for 30 sec - exiting 22:12:22 (7838): No heartbeat from core client for 30 sec - exiting 22:12:23 (7838): No heartbeat from core client for 30 sec - exiting 22:12:24 (7838): No heartbeat from core client for 30 sec - exiting 22:12:25 (7838): No heartbeat from core client for 30 sec - exiting 22:12:26 (7838): No heartbeat from core client for 30 sec - exiting 22:12:27 (7838): No heartbeat from core client for 30 sec - exiting 22:12:28 (7838): No heartbeat from core client for 30 sec - exiting 23:16:17 (7987): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 41 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jul 2011 22:46:34	1050772	12926942	hadcm3n_o6yp_1940_40_007266752_1	25,920	283,818	10.9498