Task 13147533

Name	hadcm3n_yank_1900_40_007346666_2
Workunit	7544096
Created	20 Jul 2011, 3:43:31 UTC
Sent	20 Jul 2011, 3:57:40 UTC
Report deadline	19 Oct 2011, 11:24:51 UTC
Received	8 Aug 2011, 17:54:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1197373
Run time	15 days 0 hours 44 min 26 sec
CPU time	14 days 6 hours 36 min 25 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	1.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.59</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:42:31 (2041): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:44:28 (16077): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:45:05 (16118): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:45:41 (16144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:46:49 (16148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:01 (16190): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 01:30:25 (1374): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 01:33:43 (1383): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 01:39:32 (1392): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... 01:41:47 (1400): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... 01:45:48 (1407): No heartbeat from core client for 30 sec - exiting 01:45:56 (1407): No heartbeat from core client for 30 sec - exiting 01:45:57 (1407): No heartbeat from core client for 30 sec - exiting 01:45:58 (1407): No heartbeat from core client for 30 sec - exiting 01:45:59 (1407): No heartbeat from core client for 30 sec - exiting 01:46:00 (1407): No heartbeat from core client for 30 sec - exiting 01:46:01 (1407): No heartbeat from core client for 30 sec - exiting 01:46:02 (1407): No heartbeat from core client for 30 sec - exiting 01:46:03 (1407): No heartbeat from core client for 30 sec - exiting 01:46:04 (1407): No heartbeat from core client for 30 sec - exiting 01:46:05 (1407): No heartbeat from core client for 30 sec - exiting 01:46:06 (1407): No heartbeat from core client for 30 sec - exiting 01:46:07 (1407): No heartbeat from core client for 30 sec - exiting 01:46:08 (1407): No heartbeat from core client for 30 sec - exiting 01:46:09 (1407): No heartbeat from core client for 30 sec - exiting 01:46:10 (1407): No heartbeat from core client for 30 sec - exiting 01:46:11 (1407): No heartbeat from core client for 30 sec - exiting 01:46:12 (1407): No heartbeat from core client for 30 sec - exiting 01:46:13 (1407): No heartbeat from core client for 30 sec - exiting 01:46:14 (1407): No heartbeat from core client for 30 sec - exiting 01:46:15 (1407): No heartbeat from core client for 30 sec - exiting 01:46:16 (1407): No heartbeat from core client for 30 sec - exiting 01:46:17 (1407): No heartbeat from core client for 30 sec - exiting 01:46:18 (1407): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 01:48:05 (1407): No heartbeat from core client for 30 sec - exiting 01:48:06 (1407): No heartbeat from core client for 30 sec - exiting 01:48:07 (1407): No heartbeat from core client for 30 sec - exiting 01:48:08 (1407): No heartbeat from core client for 30 sec - exiting 01:48:09 (1407): No heartbeat from core client for 30 sec - exiting 01:48:10 (1407): No heartbeat from core client for 30 sec - exiting 01:48:11 (1407): No heartbeat from core client for 30 sec - exiting 01:48:12 (1407): No heartbeat from core client for 30 sec - exiting 01:48:13 (1407): No heartbeat from core client for 30 sec - exiting 01:48:14 (1407): No heartbeat from core client for 30 sec - exiting 01:48:15 (1407): No heartbeat from core client for 30 sec - exiting 01:48:16 (1407): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day BUFFIN: Read Failed: Input/output error BUFFIN: C I/O Error ferror - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day BUFFIN: Read Failed: Input/output error BUFFIN: C I/O Error ferror - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day 02:00:54 (1416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day 02:06:34 (1437): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: Input/output error BUFFIN: C I/O Error ferror - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 CPDN Monitor - No 'heartbeat' from BOINC... 02:07:59 (1514): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:09:38 (1514): No heartbeat from core client for 30 sec - exiting 02:09:39 (1514): No heartbeat from core client for 30 sec - exiting 02:09:40 (1514): No heartbeat from core client for 30 sec - exiting 02:09:41 (1514): No heartbeat from core client for 30 sec - exiting 02:09:42 (1514): No heartbeat from core client for 30 sec - exiting 02:09:43 (1514): No heartbeat from core client for 30 sec - exiting 02:09:44 (1514): No heartbeat from core client for 30 sec - exiting 02:09:45 (1514): No heartbeat from core client for 30 sec - exiting 02:09:46 (1514): No heartbeat from core client for 30 sec - exiting 02:09:47 (1514): No heartbeat from core client for 30 sec - exiting 02:09:48 (1514): No heartbeat from core client for 30 sec - exiting 02:09:49 (1514): No heartbeat from core client for 30 sec - exiting 02:09:50 (1514): No heartbeat from core client for 30 sec - exiting 02:09:51 (1514): No heartbeat from core client for 30 sec - exiting 02:09:52 (1514): No heartbeat from core client for 30 sec - exiting 02:09:53 (1514): No heartbeat from core client for 30 sec - exiting 02:09:54 (1514): No heartbeat from core client for 30 sec - exiting 02:09:55 (1514): No heartbeat from core client for 30 sec - exiting 02:10:27 (1514): No heartbeat from core client for 30 sec - exiting 02:10:28 (1514): No heartbeat from core client for 30 sec - exiting 02:10:29 (1514): No heartbeat from core client for 30 sec - exiting 02:10:30 (1514): No heartbeat from core client for 30 sec - exiting 02:10:31 (1514): No heartbeat from core client for 30 sec - exiting 02:10:32 (1514): No heartbeat from core client for 30 sec - exiting 02:10:33 (1514): No heartbeat from core client for 30 sec - exiting 02:10:34 (1514): No heartbeat from core client for 30 sec - exiting 02:10:35 (1514): No heartbeat from core client for 30 sec - exiting 02:10:36 (1514): No heartbeat from core client for 30 sec - exiting 02:10:37 (1514): No heartbeat from core client for 30 sec - exiting 02:10:38 (1514): No heartbeat from core client for 30 sec - exiting 02:10:39 (1514): No heartbeat from core client for 30 sec - exiting 02:10:40 (1514): No heartbeat from core client for 30 sec - exiting 02:10:41 (1514): No heartbeat from core client for 30 sec - exiting 02:10:42 (1514): No heartbeat from core client for 30 sec - exiting 02:10:43 (1514): No heartbeat from core client for 30 sec - exiting 02:10:44 (1514): No heartbeat from core client for 30 sec - exiting 02:10:45 (1514): No heartbeat from core client for 30 sec - exiting 02:10:46 (1514): No heartbeat from core client for 30 sec - exiting 02:10:47 (1514): No heartbeat from core client for 30 sec - exiting 02:10:48 (1514): No heartbeat from core client for 30 sec - exiting 02:10:49 (1514): No heartbeat from core client for 30 sec - exiting 02:10:50 (1514): No heartbeat from core client for 30 sec - exiting 02:10:51 (1514): No heartbeat from core client for 30 sec - exiting 02:10:52 (1514): No heartbeat from core client for 30 sec - exiting 02:10:53 (1514): No heartbeat from core client for 30 sec - exiting 02:10:54 (1514): No heartbeat from core client for 30 sec - exiting 02:10:55 (1514): No heartbeat from core client for 30 sec - exiting 02:10:56 (1514): No heartbeat from core client for 30 sec - exiting 02:10:57 (1514): No heartbeat from core client for 30 sec - exiting 02:10:58 (1514): No heartbeat from core client for 30 sec - exiting 02:10:59 (1514): No heartbeat from core client for 30 sec - exiting 02:11:01 (1514): No heartbeat from core client for 30 sec - exiting 02:11:02 (1514): No heartbeat from core client for 30 sec - exiting 02:11:03 (1514): No heartbeat from core client for 30 sec - exiting 02:11:03 (1514): No heartbeat from core client for 30 sec - exiting 02:11:05 (1514): No heartbeat from core client for 30 sec - exiting 02:11:06 (1514): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... 02:11:37 (1628): No heartbeat from core client for 30 sec - exiting 02:11:38 (1628): No heartbeat from core client for 30 sec - exiting 02:11:39 (1628): No heartbeat from core client for 30 sec - exiting 02:11:40 (1628): No heartbeat from core client for 30 sec - exiting 02:11:41 (1628): No heartbeat from core client for 30 sec - exiting 02:11:42 (1628): No heartbeat from core client for 30 sec - exiting 02:11:43 (1628): No heartbeat from core client for 30 sec - exiting 02:11:44 (1628): No heartbeat from core client for 30 sec - exiting 02:11:45 (1628): No heartbeat from core client for 30 sec - exiting 02:11:46 (1628): No heartbeat from core client for 30 sec - exiting 02:11:47 (1628): No heartbeat from core client for 30 sec - exiting 02:11:48 (1628): No heartbeat from core client for 30 sec - exiting 02:11:49 (1628): No heartbeat from core client for 30 sec - exiting 02:11:50 (1628): No heartbeat from core client for 30 sec - exiting 02:11:51 (1628): No heartbeat from core client for 30 sec - exiting 02:11:52 (1628): No heartbeat from core client for 30 sec - exiting 02:11:53 (1628): No heartbeat from core client for 30 sec - exiting 02:12:49 (1628): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:13:33 (1628): No heartbeat from core client for 30 sec - exiting 02:13:34 (1628): No heartbeat from core client for 30 sec - exiting 02:13:35 (1628): No heartbeat from core client for 30 sec - exiting 02:13:36 (1628): No heartbeat from core client for 30 sec - exiting 02:13:37 (1628): No heartbeat from core client for 30 sec - exiting 02:13:38 (1628): No heartbeat from core client for 30 sec - exiting 02:13:39 (1628): No heartbeat from core client for 30 sec - exiting 02:13:40 (1628): No heartbeat from core client for 30 sec - exiting 02:13:41 (1628): No heartbeat from core client for 30 sec - exiting 02:13:42 (1628): No heartbeat from core client for 30 sec - exiting 02:13:43 (1628): No heartbeat from core client for 30 sec - exiting 02:13:44 (1628): No heartbeat from core client for 30 sec - exiting 02:13:45 (1628): No heartbeat from core client for 30 sec - exiting 02:13:46 (1628): No heartbeat from core client for 30 sec - exiting 02:13:47 (1628): No heartbeat from core client for 30 sec - exiting 02:13:48 (1628): No heartbeat from core client for 30 sec - exiting 02:13:49 (1628): No heartbeat from core client for 30 sec - exiting 02:13:50 (1628): No heartbeat from core client for 30 sec - exiting 02:13:51 (1628): No heartbeat from core client for 30 sec - exiting 02:13:52 (1628): No heartbeat from core client for 30 sec - exiting 02:13:53 (1628): No heartbeat from core client for 30 sec - exiting 02:13:54 (1628): No heartbeat from core client for 30 sec - exiting 02:13:55 (1628): No heartbeat from core client for 30 sec - exiting 02:13:56 (1628): No heartbeat from core client for 30 sec - exiting 02:16:27 (1628): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... 02:17:45 (1739): No heartbeat from core client for 30 sec - exiting 02:17:46 (1739): No heartbeat from core client for 30 sec - exiting 02:17:47 (1739): No heartbeat from core client for 30 sec - exiting 02:17:48 (1739): No heartbeat from core client for 30 sec - exiting 02:17:49 (1739): No heartbeat from core client for 30 sec - exiting 02:17:50 (1739): No heartbeat from core client for 30 sec - exiting 02:17:51 (1739): No heartbeat from core client for 30 sec - exiting 02:17:52 (1739): No heartbeat from core client for 30 sec - exiting 02:17:53 (1739): No heartbeat from core client for 30 sec - exiting 02:17:54 (1739): No heartbeat from core client for 30 sec - exiting 02:17:55 (1739): No heartbeat from core client for 30 sec - exiting 02:17:56 (1739): No heartbeat from core client for 30 sec - exiting 02:17:57 (1739): No heartbeat from core client for 30 sec - exiting 02:17:58 (1739): No heartbeat from core client for 30 sec - exiting 02:17:59 (1739): No heartbeat from core client for 30 sec - exiting 02:18:00 (1739): No heartbeat from core client for 30 sec - exiting 02:18:01 (1739): No heartbeat from core client for 30 sec - exiting 02:18:02 (1739): No heartbeat from core client for 30 sec - exiting 02:18:03 (1739): No heartbeat from core client for 30 sec - exiting 02:18:04 (1739): No heartbeat from core client for 30 sec - exiting 02:18:05 (1739): No heartbeat from core client for 30 sec - exiting 02:18:06 (1739): No heartbeat from core client for 30 sec - exiting 02:18:07 (1739): No heartbeat from core client for 30 sec - exiting 02:18:08 (1739): No heartbeat from core client for 30 sec - exiting 02:18:09 (1739): No heartbeat from core client for 30 sec - exiting 02:18:10 (1739): No heartbeat from core client for 30 sec - exiting 02:18:11 (1739): No heartbeat from core client for 30 sec - exiting 02:18:12 (1739): No heartbeat from core client for 30 sec - exiting 02:18:13 (1739): No heartbeat from core client for 30 sec - exiting 02:18:14 (1739): No heartbeat from core client for 30 sec - exiting 02:18:15 (1739): No heartbeat from core client for 30 sec - exiting 02:18:16 (1739): No heartbeat from core client for 30 sec - exiting 02:18:17 (1739): No heartbeat from core client for 30 sec - exiting 02:18:18 (1739): No heartbeat from core client for 30 sec - exiting 02:18:19 (1739): No heartbeat from core client for 30 sec - exiting 02:18:20 (1739): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:20:15 (1739): No heartbeat from core client for 30 sec - exiting 02:20:17 (1739): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day BUFFIN: Read Failed: Input/output error BUFFIN: C I/O Error ferror - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 02:28:57 (1765): No heartbeat from core client for 30 sec - exiting 02:28:58 (1765): No heartbeat from core client for 30 sec - exiting 02:28:59 (1765): No heartbeat from core client for 30 sec - exiting 02:29:00 (1765): No heartbeat from core client for 30 sec - exiting 02:29:01 (1765): No heartbeat from core client for 30 sec - exiting 02:29:02 (1765): No heartbeat from core client for 30 sec - exiting 02:29:03 (1765): No heartbeat from core client for 30 sec - exiting 02:29:04 (1765): No heartbeat from core client for 30 sec - exiting 02:29:05 (1765): No heartbeat from core client for 30 sec - exiting 02:29:06 (1765): No heartbeat from core client for 30 sec - exiting 02:29:07 (1765): No heartbeat from core client for 30 sec - exiting 02:29:08 (1765): No heartbeat from core client for 30 sec - exiting 02:29:09 (1765): No heartbeat from core client for 30 sec - exiting 02:29:10 (1765): No heartbeat from core client for 30 sec - exiting 02:29:11 (1765): No heartbeat from core client for 30 sec - exiting 02:29:12 (1765): No heartbeat from core client for 30 sec - exiting 02:29:13 (1765): No heartbeat from core client for 30 sec - exiting 02:29:14 (1765): No heartbeat from core client for 30 sec - exiting 02:29:15 (1765): No heartbeat from core client for 30 sec - exiting 02:29:16 (1765): No heartbeat from core client for 30 sec - exiting 02:29:17 (1765): No heartbeat from core client for 30 sec - exiting 02:29:18 (1765): No heartbeat from core client for 30 sec - exiting 02:29:19 (1765): No heartbeat from core client for 30 sec - exiting 02:29:20 (1765): No heartbeat from core client for 30 sec - exiting 02:29:21 (1765): No heartbeat from core client for 30 sec - exiting 02:29:22 (1765): No heartbeat from core client for 30 sec - exiting 02:29:23 (1765): No heartbeat from core client for 30 sec - exiting 02:29:24 (1765): No heartbeat from core client for 30 sec - exiting 02:29:25 (1765): No heartbeat from core client for 30 sec - exiting 02:29:26 (1765): No heartbeat from core client for 30 sec - exiting 02:29:27 (1765): No heartbeat from core client for 30 sec - exiting 02:29:28 (1765): No heartbeat from core client for 30 sec - exiting 02:29:29 (1765): No heartbeat from core client for 30 sec - exiting 02:29:30 (1765): No heartbeat from core client for 30 sec - exiting 02:29:31 (1765): No heartbeat from core client for 30 sec - exiting 02:29:32 (1765): No heartbeat from core client for 30 sec - exiting 02:29:33 (1765): No heartbeat from core client for 30 sec - exiting 02:29:34 (1765): No heartbeat from core client for 30 sec - exiting 02:29:35 (1765): No heartbeat from core client for 30 sec - exiting 02:29:36 (1765): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:29:37 (1765): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day BUFFIN: Read Failed: Input/output error BUFFIN: C I/O Error ferror - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:39:29 (1792): No heartbeat from core client for 30 sec - exiting 02:40:12 (1792): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... 02:41:23 (1811): No heartbeat from core client for 30 sec - exiting 02:42:57 (1811): No heartbeat from core client for 30 sec - exiting 02:42:58 (1811): No heartbeat from core client for 30 sec - exiting 02:42:59 (1811): No heartbeat from core client for 30 sec - exiting 02:43:00 (1811): No heartbeat from core client for 30 sec - exiting 02:43:01 (1811): No heartbeat from core client for 30 sec - exiting 02:43:02 (1811): No heartbeat from core client for 30 sec - exiting 02:43:03 (1811): No heartbeat from core client for 30 sec - exiting 02:43:04 (1811): No heartbeat from core client for 30 sec - exiting 02:43:05 (1811): No heartbeat from core client for 30 sec - exiting 02:43:06 (1811): No heartbeat from core client for 30 sec - exiting 02:43:07 (1811): No heartbeat from core client for 30 sec - exiting 02:43:08 (1811): No heartbeat from core client for 30 sec - exiting 02:43:09 (1811): No heartbeat from core client for 30 sec - exiting 02:43:10 (1811): No heartbeat from core client for 30 sec - exiting 02:43:11 (1811): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:43:12 (1811): No heartbeat from core client for 30 sec - exiting 02:44:34 (1811): No heartbeat from core client for 30 sec - exiting 02:44:45 (1811): No heartbeat from core client for 30 sec - exiting 02:45:19 (1811): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... 02:47:27 (1831): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:48:24 (1831): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day BUFFIN: Read Failed: Input/output error BUFFIN: C I/O Error ferror - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 02:55:47 (1839): No heartbeat from core client for 30 sec - exiting 02:55:48 (1839): No heartbeat from core client for 30 sec - exiting 02:55:49 (1839): No heartbeat from core client for 30 sec - exiting 02:55:50 (1839): No heartbeat from core client for 30 sec - exiting 02:55:51 (1839): No heartbeat from core client for 30 sec - exiting 02:55:52 (1839): No heartbeat from core client for 30 sec - exiting 02:55:53 (1839): No heartbeat from core client for 30 sec - exiting 02:55:54 (1839): No heartbeat from core client for 30 sec - exiting 02:55:55 (1839): No heartbeat from core client for 30 sec - exiting 02:55:56 (1839): No heartbeat from core client for 30 sec - exiting 02:55:57 (1839): No heartbeat from core client for 30 sec - exiting 02:55:58 (1839): No heartbeat from core client for 30 sec - exiting 02:55:59 (1839): No heartbeat from core client for 30 sec - exiting 02:56:00 (1839): No heartbeat from core client for 30 sec - exiting 02:56:01 (1839): No heartbeat from core client for 30 sec - exiting 02:56:02 (1839): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day 02:56:26 (1839): No heartbeat from core client for 30 sec - exiting 02:56:27 (1839): No heartbeat from core client for 30 sec - exiting 02:56:28 (1839): No heartbeat from core client for 30 sec - exiting 02:56:29 (1839): No heartbeat from core client for 30 sec - exiting 02:56:30 (1839): No heartbeat from core client for 30 sec - exiting 02:56:31 (1839): No heartbeat from core client for 30 sec - exiting 02:56:32 (1839): No heartbeat from core client for 30 sec - exiting 02:56:50 (1839): No heartbeat from core client for 30 sec - exiting 02:57:13 (1839): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/atmos_restart.day cpdnmonitor: error reading file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yank_1900_40_007346666/dataout/ocean_restart.day BUFFIN: Read Failed: Input/output error BUFFIN: C I/O Error ferror - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Aug 2011 01:32:16	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	233,280	1,227,298	5.2611
04 Aug 2011 10:49:06	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	207,360	1,087,215	5.2431
02 Aug 2011 17:53:46	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	181,440	947,756	5.2235
01 Aug 2011 01:13:13	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	155,520	808,066	5.1959
30 Jul 2011 07:57:59	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	129,600	668,447	5.1578
29 Jul 2011 02:50:36	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	103,680	532,016	5.1313
27 Jul 2011 03:02:09	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	77,760	403,386	5.1876
25 Jul 2011 23:00:14	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	51,840	274,449	5.2942
25 Jul 2011 19:28:08	1158954	13147533	hadcm3n_yank_1900_40_007346666_2	25,920	140,407	5.4169