Task 15494900

Name	hadcm3n_3jzp_1940_40_008265684_0
Workunit	8420808
Created	21 Dec 2012, 13:31:38 UTC
Sent	22 Dec 2012, 18:47:24 UTC
Report deadline	24 Mar 2013, 2:14:35 UTC
Received	17 Mar 2013, 0:27:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	977548
Run time	10 days 11 hours 29 min 26 sec
CPU time	9 days 11 hours 18 min 8 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.78 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3jzpko.pje1c10 Error converting file to netcdf: dataout/3jzpko.pie1c10 Error converting file to netcdf: dataout/3jzpko.pfe1c10 Error converting file to netcdf: dataout/3jzpka.phe1c10 Error converting file to netcdf: dataout/3jzpka.pge1c10 Error converting file to netcdf: dataout/3jzpka.pee1c10 Error converting file to netcdf: dataout/3jzpka.pde1c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:03:40 (6160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3jzpko.pje7c10 Error converting file to netcdf: dataout/3jzpko.pie7c10 Error converting file to netcdf: dataout/3jzpko.pfe7c10 Error converting file to netcdf: dataout/3jzpka.phe7c10 Error converting file to netcdf: dataout/3jzpka.pge7c10 Error converting file to netcdf: dataout/3jzpka.pee7c10 Error converting file to netcdf: dataout/3jzpka.pde7c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 21:06:55 (7568): No heartbeat from core client for 30 sec - exiting 21:06:56 (7568): No heartbeat from core client for 30 sec - exiting 21:06:58 (7568): No heartbeat from core client for 30 sec - exiting 21:07:02 (7568): No heartbeat from core client for 30 sec - exiting 21:07:03 (7568): No heartbeat from core client for 30 sec - exiting 21:07:06 (7568): No heartbeat from core client for 30 sec - exiting 21:07:07 (7568): No heartbeat from core client for 30 sec - exiting 21:07:08 (7568): No heartbeat from core client for 30 sec - exiting 21:07:10 (7568): No heartbeat from core client for 30 sec - exiting 21:07:11 (7568): No heartbeat from core client for 30 sec - exiting 21:07:12 (7568): No heartbeat from core client for 30 sec - exiting 21:07:14 (7568): No heartbeat from core client for 30 sec - exiting 21:07:15 (7568): No heartbeat from core client for 30 sec - exiting 21:07:18 (7568): No heartbeat from core client for 30 sec - exiting 21:07:19 (7568): No heartbeat from core client for 30 sec - exiting 21:07:22 (7568): No heartbeat from core client for 30 sec - exiting 21:07:24 (7568): No heartbeat from core client for 30 sec - exiting 21:07:27 (7568): No heartbeat from core client for 30 sec - exiting 21:07:28 (7568): No heartbeat from core client for 30 sec - exiting 21:07:31 (7568): No heartbeat from core client for 30 sec - exiting 21:07:32 (7568): No heartbeat from core client for 30 sec - exiting 21:07:34 (7568): No heartbeat from core client for 30 sec - exiting 21:07:35 (7568): No heartbeat from core client for 30 sec - exiting 21:07:38 (7568): No heartbeat from core client for 30 sec - exiting 21:07:39 (7568): No heartbeat from core client for 30 sec - exiting 21:07:43 (7568): No heartbeat from core client for 30 sec - exiting 21:07:47 (7568): No heartbeat from core client for 30 sec - exiting 21:07:48 (7568): No heartbeat from core client for 30 sec - exiting 21:07:49 (7568): No heartbeat from core client for 30 sec - exiting 21:07:51 (7568): No heartbeat from core client for 30 sec - exiting 21:07:52 (7568): No heartbeat from core client for 30 sec - exiting 21:07:55 (7568): No heartbeat from core client for 30 sec - exiting 21:07:56 (7568): No heartbeat from core client for 30 sec - exiting 21:07:57 (7568): No heartbeat from core client for 30 sec - exiting 21:07:59 (7568): No heartbeat from core client for 30 sec - exiting 21:08:00 (7568): No heartbeat from core client for 30 sec - exiting 21:08:01 (7568): No heartbeat from core client for 30 sec - exiting 21:08:03 (7568): No heartbeat from core client for 30 sec - exiting 21:08:04 (7568): No heartbeat from core client for 30 sec - exiting 21:08:05 (7568): No heartbeat from core client for 30 sec - exiting 21:08:07 (7568): No heartbeat from core client for 30 sec - exiting 21:08:11 (7568): No heartbeat from core client for 30 sec - exiting 21:08:12 (7568): No heartbeat from core client for 30 sec - exiting 21:08:13 (7568): No heartbeat from core client for 30 sec - exiting 21:08:15 (7568): No heartbeat from core client for 30 sec - exiting 21:08:16 (7568): No heartbeat from core client for 30 sec - exiting 21:08:17 (7568): No heartbeat from core client for 30 sec - exiting 21:08:19 (7568): No heartbeat from core client for 30 sec - exiting 21:08:20 (7568): No heartbeat from core client for 30 sec - exiting 21:08:21 (7568): No heartbeat from core client for 30 sec - exiting 21:08:23 (7568): No heartbeat from core client for 30 sec - exiting 21:08:24 (7568): No heartbeat from core client for 30 sec - exiting 21:08:25 (7568): No heartbeat from core client for 30 sec - exiting 21:08:27 (7568): No heartbeat from core client for 30 sec - exiting 21:08:28 (7568): No heartbeat from core client for 30 sec - exiting 21:08:29 (7568): No heartbeat from core client for 30 sec - exiting 21:08:32 (7568): No heartbeat from core client for 30 sec - exiting 21:08:33 (7568): No heartbeat from core client for 30 sec - exiting 21:08:36 (7568): No heartbeat from core client for 30 sec - exiting 21:08:37 (7568): No heartbeat from core client for 30 sec - exiting 21:08:40 (7568): No heartbeat from core client for 30 sec - exiting 21:08:41 (7568): No heartbeat from core client for 30 sec - exiting 21:08:44 (7568): No heartbeat from core client for 30 sec - exiting 21:08:45 (7568): No heartbeat from core client for 30 sec - exiting 21:08:48 (7568): No heartbeat from core client for 30 sec - exiting 21:08:49 (7568): No heartbeat from core client for 30 sec - exiting 21:08:52 (7568): No heartbeat from core client for 30 sec - exiting 21:08:53 (7568): No heartbeat from core client for 30 sec - exiting 21:08:54 (7568): No heartbeat from core client for 30 sec - exiting 21:08:56 (7568): No heartbeat from core client for 30 sec - exiting 21:08:57 (7568): No heartbeat from core client for 30 sec - exiting 21:08:58 (7568): No heartbeat from core client for 30 sec - exiting 21:09:00 (7568): No heartbeat from core client for 30 sec - exiting 21:09:01 (7568): No heartbeat from core client for 30 sec - exiting 21:09:02 (7568): No heartbeat from core client for 30 sec - exiting 21:09:04 (7568): No heartbeat from core client for 30 sec - exiting 21:09:05 (7568): No heartbeat from core client for 30 sec - exiting 21:09:06 (7568): No heartbeat from core client for 30 sec - exiting 21:09:08 (7568): No heartbeat from core client for 30 sec - exiting 21:09:09 (7568): No heartbeat from core client for 30 sec - exiting 21:09:10 (7568): No heartbeat from core client for 30 sec - exiting 21:09:12 (7568): No heartbeat from core client for 30 sec - exiting 21:09:13 (7568): No heartbeat from core client for 30 sec - exiting 21:09:15 (7568): No heartbeat from core client for 30 sec - exiting 21:09:17 (7568): No heartbeat from core client for 30 sec - exiting 21:09:18 (7568): No heartbeat from core client for 30 sec - exiting 21:09:19 (7568): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day 21:09:21 (7568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:09:22 (7568): No heartbeat from core client for 30 sec - exiting 21:09:26 (7568):21:12:16 (7264): No heartbeat from core client for 30 sec - exiting 21:12:17 (7264): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day 21:12:18 (7264): No heartbeat from core client for 30 sec - exiting 21:12:19 (7264): No heartbeat from core client for 30 sec - exiting 21:12:20 (7264): No heartbeat from core client for 30 sec - exiting 21:12:21 (7264): No heartbeat from core client for 30 sec - exiting 21:12:22 (7264): No heartbeat from core client for 30 sec - exiting 21:12:24 (7264): No heartbeat from core client for 30 sec - exiting 21:12:25 (7264): No heartbeat from core client for 30 sec - exiting forrtl: Access is denied. CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 21:24:36 (196): No heartbeat from core client for 30 sec - exiting 21:24:37 (196): No heartbeat from core client for 30 sec - exiting 21:24:38 (196): No heartbeat from core client for 30 sec - exiting 21:24:39 (196): No heartbeat from core client for 30 sec - exiting 21:24:40 (196): No heartbeat from core client for 30 sec - exiting 21:24:41 (196): No heartbeat from core client for 30 sec - exiting 21:24:42 (196): No heartbeat from core client for 30 sec - exiting 21:24:45 (196): No heartbeat from core client for 30 sec - exiting 21:24:46 (196): No heartbeat from core client for 30 sec - exiting 21:24:49 (196): No heartbeat from core client for 30 sec - exiting 21:24:50 (196): No heartbeat from core client for 30 sec - exiting 21:24:53 (196): No heartbeat from core client for 30 sec - exiting 21:24:54 (196): No heartbeat from core client for 30 sec - exiting 21:24:57 (196): No heartbeat from core client for 30 sec - exiting 21:24:58 (196): No heartbeat from core client for 30 sec - exiting 21:25:01 (196): No heartbeat from core client for 30 sec - exiting 21:25:02 (196): No heartbeat from core client for 30 sec - exiting 21:25:05 (196): No heartbeat from core client for 30 sec - exiting 21:25:06 (196): No heartbeat from core client for 30 sec - exiting 21:25:10 (196): No heartbeat from core client for 30 sec - exiting 21:25:11 (196): No heartbeat from core client for 30 sec - exiting 21:25:14 (196): No heartbeat from core client for 30 sec - exiting 21:25:17 (196): No heartbeat from core client for 30 sec - exiting 21:25:18 (196): No heartbeat from core client for 30 sec - exiting 21:25:21 (196): No heartbeat from core client for 30 sec - exiting 21:25:22 (196): No heartbeat from core client for 30 sec - exiting 21:25:25 (196): No heartbeat from core client for 30 sec - exiting 21:25:27 (196): No heartbeat from core client for 30 sec - exiting 21:25:30 (196): No heartbeat from core client for 30 sec - exiting 21:25:31 (196): No heartbeat from core client for 30 sec - exiting 21:25:34 (196): No heartbeat from core client for 30 sec - exiting 21:25:35 (196): No heartbeat from core client for 30 sec - exiting 21:25:38 (196): No heartbeat from core client for 30 sec - exiting 21:25:39 (196): No heartbeat from core client for 30 sec - exiting 21:25:40 (196): No heartbeat from core client for 30 sec - exiting 21:25:42 (196): No heartbeat from core client for 30 sec - exiting 21:25:43 (196): No heartbeat from core client for 30 sec - exiting 21:25:44 (196): No heartbeat from core client for 30 sec - exiting 21:25:46 (196): No heartbeat from core client for 30 sec - exiting 21:25:49 (196): No heartbeat from core client for 30 sec - exiting 21:25:50 (196): No heartbeat from core client for 30 sec - exiting 21:25:53 (196): No heartbeat from core client for 30 sec - exiting 21:25:54 (196): No heartbeat from core client for 30 sec - exiting 21:25:57 (196): No heartbeat from core client for 30 sec - exiting 21:25:59 (196): No heartbeat from core client for 30 sec - exiting 21:26:02 (196): No heartbeat from core client for 30 sec - exiting 21:26:03 (196): No heartbeat from core client for 30 sec - exiting 21:26:06 (196): No heartbeat from core client for 30 sec - exiting 21:26:07 (196): No heartbeat from core client for 30 sec - exiting 21:26:10 (196): No heartbeat from core client for 30 sec - exiting 21:26:11 (196): No heartbeat from core client for 30 sec - exiting 21:26:12 (196): No heartbeat from core client for 30 sec - exiting 21:26:14 (196): No heartbeat from core client for 30 sec - exiting 21:26:15 (196): No heartbeat from core client for 30 sec - exiting 21:26:16 (196): No heartbeat from core client for 30 sec - exiting 21:26:19 (196): No heartbeat from core client for 30 sec - exiting 21:26:20 (196): No heartbeat from core client for 30 sec - exiting 21:26:23 (196): No heartbeat from core client for 30 sec - exiting 21:26:24 (196): No heartbeat from core client for 30 sec - exiting 21:26:26 (196): No heartbeat from core client for 30 sec - exiting 21:26:27 (196): No heartbeat from core client for 30 sec - exiting 21:26:28 (196): No heartbeat from core client for 30 sec - exiting 21:26:30 (196): No heartbeat from core client for 30 sec - exiting 21:26:31 (196): No heartbeat from core client for 30 sec - exiting 21:26:32 (196): No heartbeat from core client for 30 sec - exiting 21:26:34 (196): No heartbeat from core client for 30 sec - exiting 21:26:35 (196): No heartbeat from core client for 30 sec - exiting 21:26:37 (196): No heartbeat from core client for 30 sec - exiting 21:26:38 (196): No heartbeat from core client for 30 sec - exiting 21:26:40 (196): No heartbeat from core client for 30 sec - exiting 21:26:41 (196): No heartbeat from core client for 30 sec - exiting 21:26:43 (196): No heartbeat from core client for 30 sec - exiting 21:26:44 (196): No heartbeat from core client for 30 sec - exiting 21:26:45 (196): No heartbeat from core client for 30 sec - exiting 21:26:47 (196): No heartbeat from core client for 30 sec - exiting 21:26:48 (196): No heartbeat from core client for 30 sec - exiting 21:26:51 (196): No heartbeat from core client for 30 sec - exiting 21:26:52 (196): No heartbeat from core client for 30 sec - exiting 21:26:55 (196): No heartbeat from core client for 30 sec - exiting 21:26:56 (196): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day 22:33:23 (3392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jzp_1940_40_008265684/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:17:33 (6624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:42 (2868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:46:38 (5976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Mar 2013 00:32:09	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	259,200	818,280	3.1569
22 Jan 2013 00:54:01	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	233,280	395,906	1.6971
20 Jan 2013 18:33:20	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	207,360	352,377	1.6993
18 Jan 2013 03:58:45	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	181,440	309,320	1.7048
13 Jan 2013 03:48:41	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	155,520	265,413	1.7066
12 Jan 2013 01:58:01	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	129,600	221,774	1.7112
06 Jan 2013 20:06:19	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	103,680	178,097	1.7178
03 Jan 2013 04:30:03	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	77,760	134,093	1.7244
26 Dec 2012 05:43:21	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	51,840	88,953	1.7159
23 Dec 2012 21:15:51	977548	15494900	hadcm3n_3jzp_1940_40_008265684_0	25,920	45,132	1.7412