Task 15870633

Name	hadcm3n_o107_1980_40_008384046_2
Workunit	8534905
Created	28 Jun 2013, 7:44:31 UTC
Sent	2 Jul 2013, 1:27:27 UTC
Report deadline	1 Oct 2013, 8:54:38 UTC
Received	27 Aug 2013, 12:06:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1037154
Run time	15 days 4 hours 25 min 13 sec
CPU time	3 days 0 hours 11 min 52 sec
Validate state	Invalid
Credit	2,177.28
Device peak FLOPS	2.24 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file dataout/ocean_restart.day BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. CPDN Monitor - Quit request from BOINC... cpdnmonitor: error reading file dataout/ocean_restart.day 16:43:37 (1980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:38 (1980): No heartbeat from core client for 30 sec - exiting 16:43:39 (1980): No heartbeat from core client for 30 sec - exiting 16:44:19 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:04:10 (2916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:32 (2252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:07:48 (2788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:45:08 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:58 (4076): No heartbeat from core client for 30 sec - exiting 02:00:00 (4076): No heartbeat from core client for 30 sec - exiting 06:31:23 (3540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:32:10 (3540): No heartbeat from core client for 30 sec - exiting 06:42:14 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:59 (748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:26:00 (748): No heartbeat from core client for 30 sec - exiting 09:33:07 (1848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:57:21 (3948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:57:22 (3948): No heartbeat from core client for 30 sec - exiting 14:58:58 (1596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 forrtl: There is not enough space on the disk. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1624, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 15:00:01 (2800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:00:03 (2800): No heartbeat from core client for 30 sec - exiting 15:00:04 (2800): No heartbeat from core client for 30 sec - exiting 15:00:05 (2800): No heartbeat from core client for 30 sec - exiting 15:00:50 (2644): No heartbeat from core client for 30 sec - exiting 15:00:51 (2644): No heartbeat from core client for 30 sec - exiting 15:00:52 (2644): No heartbeat from core client for 30 sec - exiting 15:00:53 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:08:25 (2584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:22:56 (492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:22:58 (492): No heartbeat from core client for 30 sec - exiting 13:23:03 (492): No heartbeat from core client for 30 sec - exiting 13:23:04 (492): No heartbeat from core client for 30 sec - exiting 13:23:06 (492): No heartbeat from core client for 30 sec - exiting 13:25:15 (964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:36:20 (2488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:36:26 (2488): No heartbeat from core client for 30 sec - exiting 13:36:27 (2488): No heartbeat from core client for 30 sec - exiting 14:14:13 (2896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:14:15 (2896): No heartbeat from core client for 30 sec - exiting 14:14:16 (2896): No heartbeat from core client for 30 sec - exiting 14:14:17 (2896): No heartbeat from core client for 30 sec - exiting 14:15:00 (3612): No heartbeat from core client for 30 sec - exiting 14:15:01 (3612): No heartbeat from core client for 30 sec - exiting 14:15:02 (3612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:13:03 (2940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:13:04 (2940): No heartbeat from core client for 30 sec - exiting 15:13:05 (2940): No heartbeat from core client for 30 sec - exiting 15:13:07 (2940): No heartbeat from core client for 30 sec - exiting 15:13:08 (2940): No heartbeat from core client for 30 sec - exiting 15:13:09 (2940): No heartbeat from core client for 30 sec - exiting 15:13:10 (2940): No heartbeat from core client for 30 sec - exiting 15:14:01 (1760): No heartbeat from core client for 30 sec - exiting 15:14:02 (1760): No heartbeat from core client for 30 sec - exiting 15:14:03 (1760): No heartbeat from core client for 30 sec - exiting 15:14:04 (1760): No heartbeat from core client for 30 sec - exiting 15:14:05 (1760): No heartbeat from core client for 30 sec - exiting 15:14:06 (1760): No heartbeat from core client for 30 sec - exiting 15:14:21 (1760): No heartbeat from core client for 30 sec - exiting 15:14:22 (1760): No heartbeat from core client for 30 sec - exiting 15:14:23 (1760): No heartbeat from core client for 30 sec - exiting 15:14:24 (1760): No heartbeat from core client for 30 sec - exiting 15:14:26 (1760): No heartbeat from core client for 30 sec - exiting 15:14:27 (1760): No heartbeat from core client for 30 sec - exiting 15:14:28 (1760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:13 (2244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:14 (2244): No heartbeat from core client for 30 sec - exiting 16:00:15 (2244): No heartbeat from core client for 30 sec - exiting 16:00:16 (2244): No heartbeat from core client for 30 sec - exiting 16:00:17 (2244): No heartbeat from core client for 30 sec - exiting 16:00:18 (2244): No heartbeat from core client for 30 sec - exiting 16:00:19 (2244): No heartbeat from core client for 30 sec - exiting 16:01:58 (2000): No heartbeat from core client for 30 sec - exiting 16:01:59 (2000): No heartbeat from core client for 30 sec - exiting 16:02:00 (2000): No heartbeat from core client for 30 sec - exiting 16:02:01 (2000): No heartbeat from core client for 30 sec - exiting 16:02:02 (2000): No heartbeat from core client for 30 sec - exiting 16:02:03 (2000): No heartbeat from core client for 30 sec - exiting 16:02:04 (2000): No heartbeat from core client for 30 sec - exiting 16:03:38 (2000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:43 (1428): No heartbeat from core client for 30 sec - exiting 16:04:44 (1428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:45 (1428): No heartbeat from core client for 30 sec - exiting 16:04:46 (1428): No heartbeat from core client for 30 sec - exiting 16:04:47 (1428): No heartbeat from core client for 30 sec - exiting 16:06:51 (3228): No heartbeat from core client for 30 sec - exiting 16:06:52 (3228): No heartbeat from core client for 30 sec - exiting 16:06:53 (3228): No heartbeat from core client for 30 sec - exiting 16:06:54 (3228): No heartbeat from core client for 30 sec - exiting 16:06:55 (3228): No heartbeat from core client for 30 sec - exiting 16:06:56 (3228): No heartbeat from core client for 30 sec - exiting 16:06:58 (3228): No heartbeat from core client for 30 sec - exiting 16:06:59 (3228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:46 (3020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:47 (3020): No heartbeat from core client for 30 sec - exiting 16:10:48 (3020): No heartbeat from core client for 30 sec - exiting 16:10:49 (3020): No heartbeat from core client for 30 sec - exiting 16:10:50 (3020): No heartbeat from core client for 30 sec - exiting 16:10:52 (3020): No heartbeat from core client for 30 sec - exiting 16:21:42 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:43 (3472): No heartbeat from core client for 30 sec - exiting 16:21:45 (3472): No heartbeat from core client for 30 sec - exiting 16:21:46 (3472): No heartbeat from core client for 30 sec - exiting 16:21:47 (3472): No heartbeat from core client for 30 sec - exiting 16:21:48 (3472): No heartbeat from core client for 30 sec - exiting 16:24:04 (1996): No heartbeat from core client for 30 sec - exiting 16:24:05 (1996): No heartbeat from core client for 30 sec - exiting 16:24:06 (1996): No heartbeat from core client for 30 sec - exiting 16:24:07 (1996): No heartbeat from core client for 30 sec - exiting 16:24:08 (1996): No heartbeat from core client for 30 sec - exiting 16:24:09 (1996): No heartbeat from core client for 30 sec - exiting 16:24:11 (1996): No heartbeat from core client for 30 sec - exiting 16:24:12 (1996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:26:27 (3852): No heartbeat from core client for 30 sec - exiting 16:26:28 (3852): No heartbeat from core client for 30 sec - exiting 16:26:29 (3852): No heartbeat from core client for 30 sec - exiting 16:26:30 (3852): No heartbeat from core client for 30 sec - exiting 16:26:31 (3852): No heartbeat from core client for 30 sec - exiting 16:26:32 (3852): No heartbeat from core client for 30 sec - exiting 16:26:33 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:49:45 (3024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:50 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:02:26 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:02:27 (3472): No heartbeat from core client for 30 sec - exiting 18:05:05 (580): No heartbeat from core client for 30 sec - exiting 18:05:06 (580): No heartbeat from core client for 30 sec - exiting 18:05:07 (580): No heartbeat from core client for 30 sec - exiting 18:05:08 (580): No heartbeat from core client for 30 sec - exiting 18:05:09 (580): No heartbeat from core client for 30 sec - exiting 18:05:11 (580): No heartbeat from core client for 30 sec - exiting 18:05:12 (580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:13 (580): No heartbeat from core client for 30 sec - exiting 18:07:56 (1460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:09:03 (2252): No heartbeat from core client for 30 sec - exiting 18:09:04 (2252): No heartbeat from core client for 30 sec - exiting 18:09:05 (2252): No heartbeat from core client for 30 sec - exiting 18:09:06 (2252): No heartbeat from core client for 30 sec - exiting 18:09:07 (2252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:16:18 (2220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:16 (512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:21 (512): No heartbeat from core client for 30 sec - exiting 22:48:22 (512): No heartbeat from core client for 30 sec - exiting 22:48:24 (512): No heartbeat from core client for 30 sec - exiting 22:48:25 (512): No heartbeat from core client for 30 sec - exiting 22:48:26 (512): No heartbeat from core client for 30 sec - exiting 22:48:27 (512): No heartbeat from core client for 30 sec - exiting 22:48:28 (512): No heartbeat from core client for 30 sec - exiting 22:48:29 (512): No heartbeat from core client for 30 sec - exiting 22:49:51 (1976): No heartbeat from core client for 30 sec - exiting 22:49:52 (1976): No heartbeat from core client for 30 sec - exiting 22:49:54 (1976): No heartbeat from core client for 30 sec - exiting 22:49:55 (1976): No heartbeat from core client for 30 sec - exiting 22:49:56 (1976): No heartbeat from core client for 30 sec - exiting 22:49:58 (1976): No heartbeat from core client for 30 sec - exiting 22:50:01 (1976): No heartbeat from core client for 30 sec - exiting 22:50:04 (1976): No heartbeat from core client for 30 sec - exiting 22:50:06 (1976): No heartbeat from core client for 30 sec - exiting 22:50:07 (1976): No heartbeat from core client for 30 sec - exiting 22:50:08 (1976): No heartbeat from core client for 30 sec - exiting 22:50:09 (1976): No heartbeat from core client for 30 sec - exiting 22:50:10 (1976): No heartbeat from core client for 30 sec - exiting 22:50:11 (1976): No heartbeat from core client for 30 sec - exiting 22:50:12 (1976): No heartbeat from core client for 30 sec - exiting 22:50:14 (1976): No heartbeat from core client for 30 sec - exiting 22:50:15 (1976): No heartbeat from core client for 30 sec - exiting 22:50:16 (1976): No heartbeat from core client for 30 sec - exiting 22:50:17 (1976): No heartbeat from core client for 30 sec - exiting 22:50:18 (1976): No heartbeat from core client for 30 sec - exiting 22:50:19 (1976): No heartbeat from core client for 30 sec - exiting 22:50:20 (1976): No heartbeat from core client for 30 sec - exiting 22:50:21 (1976): No heartbeat from core client for 30 sec - exiting 22:50:22 (1976): No heartbeat from core client for 30 sec - exiting 22:50:24 (1976): No heartbeat from core client for 30 sec - exiting 22:50:25 (1976): No heartbeat from core client for 30 sec - exiting 22:50:26 (1976): No heartbeat from core client for 30 sec - exiting 22:50:27 (1976): No heartbeat from core client for 30 sec - exiting 22:50:28 (1976): No heartbeat from core client for 30 sec - exiting 22:50:29 (1976): No heartbeat from core client for 30 sec - exiting 22:50:30 (1976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:01:03 (2476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:08:20 (3944): No heartbeat from core client for 30 sec - exiting 23:08:22 (3944): No heartbeat from core client for 30 sec - exiting 23:08:23 (3944): No heartbeat from core client for 30 sec - exiting 23:08:24 (3944): No heartbeat from core client for 30 sec - exiting 23:08:25 (3944): No heartbeat from core client for 30 sec - exiting 23:08:26 (3944): No heartbeat from core client for 30 sec - exiting 23:08:27 (3944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 12:45:43 (3456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:22 (3588): No heartbeat from core client for 30 sec - exiting 12:46:23 (3588): No heartbeat from core client for 30 sec - exiting 12:46:24 (3588): No heartbeat from core client for 30 sec - exiting 12:46:25 (3588): No heartbeat from core client for 30 sec - exiting 12:55:13 (3588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:52:39 (2724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:52:41 (2724): No heartbeat from core client for 30 sec - exiting 14:52:42 (2724): No heartbeat from core client for 30 sec - exiting 16:48:23 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:48:27 (3520): No heartbeat from core client for 30 sec - exiting 16:48:29 (3520): No heartbeat from core client for 30 sec - exiting 16:49:07 (304): No heartbeat from core client for 30 sec - exiting 16:49:08 (304): No heartbeat from core client for 30 sec - exiting 16:49:09 (304): No heartbeat from core client for 30 sec - exiting 16:49:10 (304): No heartbeat from core client for 30 sec - exiting 16:49:11 (304): No heartbeat from core client for 30 sec - exiting 16:49:13 (304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:49 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:50 (4012): No heartbeat from core client for 30 sec - exiting 16:50:51 (4012): No heartbeat from core client for 30 sec - exiting 16:53:47 (2460): No heartbeat from core client for 30 sec - exiting 16:53:48 (2460): No heartbeat from core client for 30 sec - exiting 16:53:49 (2460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:01 (132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:01:05 (2136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:01:06 (2136): No heartbeat from core client for 30 sec - exiting 17:01:07 (2136): No heartbeat from core client for 30 sec - exiting 17:01:08 (2136): No heartbeat from core client for 30 sec - exiting 17:01:09 (2136): No heartbeat from core client for 30 sec - exiting 17:03:27 (2340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:03:28 (2340): No heartbeat from core client for 30 sec - exiting forrtl: There is not enough space on the disk. 14:17:48 (3988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:51 (3988): No heartbeat from core client for 30 sec - exiting 14:17:53 (3988): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day 14:19:34 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:19:35 (4748): No heartbeat from core client for 30 sec - exiting 14:19:36 (4748): No heartbeat from core client for 30 sec - exiting 14:19:37 (4748): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 14:25:56 (3692): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day 14:29:16 (5764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:17 (5764): No heartbeat from core client for 30 sec - exiting 14:29:18 (5764): No heartbeat from core client for 30 sec - exiting 14:29:19 (5764): No heartbeat from core client for 30 sec - exiting 14:29:20 (5764): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file C:\Documents and Settings\All Users\Application Data\BOINC/projects/climateprediction.net/hadcm3n_o107_1980_40_008384046/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Aug 2013 12:07:17	1037154	15870633	hadcm3n_o107_1980_40_008384046_2	181,440	229,988	1.2676
11 Jul 2013 00:52:49	1037154	15870633	hadcm3n_o107_1980_40_008384046_2	155,520	165,752	1.0658
09 Jul 2013 00:43:51	1037154	15870633	hadcm3n_o107_1980_40_008384046_2	129,600	100,861	0.7782
08 Jul 2013 01:05:41	1037154	15870633	hadcm3n_o107_1980_40_008384046_2	103,680	34,777	0.3354
06 Jul 2013 04:24:16	1037154	15870633	hadcm3n_o107_1980_40_008384046_2	77,760	34,621	0.4452
04 Jul 2013 14:05:11	1037154	15870633	hadcm3n_o107_1980_40_008384046_2	51,840	37,995	0.7329
03 Jul 2013 01:53:46	1037154	15870633	hadcm3n_o107_1980_40_008384046_2	25,920	36,748	1.4177