Task 13678600

Name	hadcm3n_yh3l_1980_40_007548553_1
Workunit	7745785
Created	30 Nov 2011, 22:38:59 UTC
Sent	30 Nov 2011, 22:42:20 UTC
Report deadline	1 Mar 2012, 6:09:31 UTC
Received	8 Jan 2012, 17:43:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1178602
Run time	20 days 12 hours 28 min 45 sec
CPU time	19 days 0 hours 33 min 8 sec
Validate state	Invalid
Credit	8,087.04
Device peak FLOPS	1.78 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13212, selfPID=13212, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on yh3lko.dak27r0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on yh3lko.dak6ao0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: Not enough storage is available to process this command. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7900, iMonCtr=1 Model crash detected, will try to restart... forrtl: Not enough storage is available to process this command. forrtl: Not enough storage is available to process this command. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7900, iMonCtr=1 Model crash detected, will try to restart... forrtl: Not enough storage is available to process this command. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7900, iMonCtr=1 Model crash detected, will try to restart... forrtl: Not enough storage is available to process this command. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7900, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_se_6.07_windows_intelx86.dll cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_um_6.07_windows_intelx86.exe cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_yh3l_1980_40_007548553/jobs/xabnk.ihist cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_yh3l_1980_40_007548553/jobs/xabnk.namelists cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_yh3l_1980_40_007548553/dataout/atmos_restart.day cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_yh3l_1980_40_007548553/dataout/ocean_restart.day no start tag in app init data 17:27:22 (7900): Can't parse init data file - running in standalone mode no start tag in app init data 17:27:23 (6096): Can't parse init data file - running in standalone mode forrtl: Not enough storage is available to process this command. Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Jan 2012 15:13:35	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	673,920	1,612,668	2.3930
06 Jan 2012 13:09:10	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	648,000	1,552,867	2.3964
05 Jan 2012 08:34:27	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	622,080	1,491,900	2.3982
04 Jan 2012 04:04:59	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	596,160	1,429,325	2.3976
03 Jan 2012 01:19:27	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	570,240	1,367,395	2.3979
02 Jan 2012 04:33:56	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	544,320	1,306,475	2.4002
01 Jan 2012 03:11:14	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	518,400	1,245,246	2.4021
31 Dec 2011 08:51:28	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	492,480	1,182,710	2.4015
30 Dec 2011 01:37:51	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	466,560	1,121,894	2.4046
29 Dec 2011 04:57:50	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	440,640	1,061,729	2.4095
28 Dec 2011 08:49:44	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	414,720	998,385	2.4074
27 Dec 2011 07:12:14	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	388,800	935,388	2.4058
26 Dec 2011 01:08:41	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	362,880	873,160	2.4062
24 Dec 2011 23:46:34	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	336,960	811,168	2.4073
23 Dec 2011 23:38:10	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	311,040	749,070	2.4083
21 Dec 2011 04:57:01	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	285,120	685,822	2.4054
18 Dec 2011 22:05:04	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	259,200	622,268	2.4007
17 Dec 2011 16:35:37	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	233,280	559,209	2.3972
16 Dec 2011 14:08:14	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	207,360	496,410	2.3940
15 Dec 2011 02:27:40	1178602	13678600	hadcm3n_yh3l_1980_40_007548553_1	181,440	433,136	2.3872