Task 14762296

Name	hadcm3n_o053_2060_40_007999296_0
Workunit	8154410
Created	31 May 2012, 19:16:37 UTC
Sent	31 May 2012, 19:25:24 UTC
Report deadline	31 Aug 2012, 2:52:35 UTC
Received	4 Jul 2012, 18:39:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1195437
Run time	8 days 19 hours 38 min 49 sec
CPU time	2 hours 41 min 25 sec
Validate state	Invalid
Credit	311.04
Device peak FLOPS	2.73 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> El dispositivo no reconoce el comando. (0x16) - exit code 22 (0x16) </message> <stderr_txt> hadcm3n_o053_2060_40_007999296/datain/masks/cpdn_omsk_s_ind_300.nc: write error (disk full?). Continue? (y/n/^C) warning: hadcm3n_o053_2060_40_007999296/datain/masks/cpdn_omsk_s_ind_300.nc is probably truncated forrtl: Espacio de almacenamiento insuficiente para procesar este comando. Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: Espacio de almacenamiento insuficiente para procesar este comando. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12028, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_um_6.07_windows_intelx86.exe forrtl: Espacio de almacenamiento insuficiente para procesar este comando. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12028, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o053_2060_40_007999296/dataout/atmos_restart.day Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file dataout/ocean_restart.day BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error ferror - Unit 116 - Return code = 32 Model crashed: REPLANCA :I/O ERROR tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... forrtl: Espacio de almacenamiento insuficiente para procesar este comando. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12856, iMonCtr=1 Model crash detected, will try to restart... forrtl: Espacio de almacenamiento insuficiente para procesar este comando. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12856, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Error converting file to netcdf: dataout/o053ko.pjq1c10 Error converting file to netcdf: dataout/o053ko.piq1c10 Error converting file to netcdf: dataout/o053ko.pfq1c10 Error converting file to netcdf: dataout/o053ka.phq1c10 Error converting file to netcdf: dataout/o053ka.pgq1c10 Error converting file to netcdf: dataout/o053ka.peq1c10 Error converting file to netcdf: dataout/o053ka.pdq1c10 MainError: 01:24:32 AM No files match the supplied pattern. MainError: 01:24:32 AM No files match the supplied pattern. Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error ferror - Unit 21 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error ferror - Unit 116 - Return code = 32 Model crashed: REPLANCA :I/O ERROR tmp/pipe_dummy 2048 cpdnmonitor: error reading file dataout/ocean_restart.day cpdnmonitor: error reading file dataout/ocean_restart.day Model crashed: TEMPHIST: Write ERROR on history file for namelistNLCFILES tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... 06:37:19 (12856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4108, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4108, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6712, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6712, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6712, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Jun 2012 01:29:25	1195437	14762296	hadcm3n_o053_2060_40_007999296_0	25,920	10,317	0.3980