Task 15502552

Name	hadcm3n_o40y_2140_40_008269685_0
Workunit	8424809
Created	23 Dec 2012, 23:48:29 UTC
Sent	24 Dec 2012, 20:35:50 UTC
Report deadline	26 Mar 2013, 4:03:01 UTC
Received	6 Feb 2013, 6:30:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1254930
Run time	24 days 10 hours 20 min 9 sec
CPU time	20 days 12 hours 20 min 39 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o40yko.day16c0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... OPEN: Unable to Open File dataout/o40yka.pdy9c10 for Read/Write Model crashed: STWORK : Error opening output PP file on unit 63 tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on o40yko.daz13i0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o40yko.daz15g0 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o40yko.daz1ch0 Ocean Restart file copy failed on o40yko.daz1cn0 Ocean Restart file copy failed on o40yko.daz1cp0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o40yko.daz72f0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error 13 - Permission denied Suspended CPDN Monitor - Suspend request from BOINC... MainError: 10:09:25 PM No files match the supplied pattern. MainError: 10:09:25 PM No files match the supplied pattern. MainError: 07:56:41 AM No files match the supplied pattern. MainError: 07:56:41 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 07:48:03 PM No files match the supplied pattern. MainError: 07:48:03 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 06:59:53 AM No files match the supplied pattern. MainError: 06:59:53 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 06:52:41 PM No files match the supplied pattern. MainError: 06:52:41 PM No files match the supplied pattern. MainError: 06:42:14 AM No files match the supplied pattern. MainError: 06:42:14 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5580, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5580, iMonCtr=1 Model crash detected, will try to restart... 21:37:25 (5580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11704, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11704, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11704, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11704, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:17:52 (31748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error -49 - Variable not found Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Suspended CPDN Monitor - Suspend request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied CPDN Monitor - Quit request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied MainError: 04:56:58 PM No files match the supplied pattern. MainError: 04:56:58 PM No files match the supplied pattern. MainError: 02:47:40 AM No files match the supplied pattern. MainError: 02:47:40 AM No files match the supplied pattern. MainError: 12:29:50 AM No files match the supplied pattern. MainError: 12:29:50 AM No files match the supplied pattern. MainError: 10:09:59 PM No files match the supplied pattern. MainError: 10:09:59 PM No files match the supplied pattern. MainError: 07:50:21 AM No files match the supplied pattern. MainError: 07:50:21 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 07:09:26 PM No files match the supplied pattern. MainError: 07:09:26 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 04:41:09 AM No files match the supplied pattern. MainError: 04:41:09 AM No files match the supplied pattern. MainError: 02:34:35 PM No files match the supplied pattern. MainError: 02:34:35 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 06:55:41 AM No files match the supplied pattern. MainError: 06:55:42 AM No files match the supplied pattern. MainError: 04:29:17 PM No files match the supplied pattern. MainError: 04:29:17 PM No files match the supplied pattern. Error converting file to netcdf: dataout/o40yka.ph11c10 Error converting file to netcdf: dataout/o40yka.pg11c10 Error converting file to netcdf: dataout/o40yka.pe11c10 MainError: 04:31:15 AM No files match the supplied pattern. MainError: 04:31:15 AM No files match the supplied pattern. BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Feb 2013 04:32:16	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	777,600	1,873,493	2.4093
05 Feb 2013 16:30:57	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	751,680	1,839,196	2.4468
05 Feb 2013 06:56:31	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	725,760	1,805,368	2.4876
04 Feb 2013 14:35:37	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	699,840	1,771,547	2.5314
04 Feb 2013 05:37:05	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	673,920	1,737,897	2.5788
16 Jan 2013 06:47:07	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	648,000	840,291	1.2967
15 Jan 2013 19:05:00	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	622,080	805,328	1.2946
15 Jan 2013 07:53:17	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	596,160	770,725	1.2928
14 Jan 2013 19:48:19	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	570,240	736,145	1.2909
14 Jan 2013 07:58:37	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	544,320	701,694	1.2891
13 Jan 2013 22:09:50	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	518,400	667,924	1.2884
13 Jan 2013 12:45:02	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	492,480	634,403	1.2882
13 Jan 2013 02:38:31	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	466,560	600,413	1.2869
11 Jan 2013 11:39:48	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	440,640	566,421	1.2855
08 Jan 2013 12:40:12	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	414,720	532,498	1.2840
08 Jan 2013 02:40:44	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	388,800	498,842	1.2830
07 Jan 2013 14:46:27	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	362,880	465,096	1.2817
06 Jan 2013 22:18:29	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	336,960	430,652	1.2781
05 Jan 2013 01:54:14	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	311,040	396,328	1.2742
04 Jan 2013 10:35:42	1254930	15502552	hadcm3n_o40y_2140_40_008269685_0	285,120	361,787	1.2689