Task 13656769

Name	hadcm3n_o786_1940_40_007536255_2
Workunit	7733487
Created	23 Nov 2011, 19:09:18 UTC
Sent	23 Nov 2011, 19:12:04 UTC
Report deadline	23 Feb 2012, 2:39:15 UTC
Received	3 Nov 2012, 16:24:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1180613
Run time	5 days 23 hours 37 min 26 sec
CPU time	5 days 23 hours 37 min 26 sec
Validate state	Invalid
Credit	1,555.20
Device peak FLOPS	1.40 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.3.19</core_client_version> <message>The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o786ko.dae3870 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=644, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o786ko.dae52s0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on o786ko.dae5490 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3280, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3188, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3188, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3188, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3188, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3188, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Program Files\Climate Change Experiment/projects/bbc.cpdn.org/hadcm3n_o786_1940_40_007536255/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3188, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Oct 2012 12:53:32	1180613	13656769	hadcm3n_o786_1940_40_007536255_2	129,600	504,921	3.8960
04 Jun 2012 19:37:13	1180613	13656769	hadcm3n_o786_1940_40_007536255_2	103,680	402,198	3.8792
15 Apr 2012 20:33:46	1180613	13656769	hadcm3n_o786_1940_40_007536255_2	77,760	303,957	3.9089
06 Mar 2012 14:58:13	1180613	13656769	hadcm3n_o786_1940_40_007536255_2	51,840	201,236	3.8819
07 Feb 2012 20:17:46	1180613	13656769	hadcm3n_o786_1940_40_007536255_2	25,920	100,994	3.8964