Task 15597668

Name	hadcm3n_4gnu_1940_40_008310379_0
Workunit	8461514
Created	8 Feb 2013, 0:39:19 UTC
Sent	8 Feb 2013, 0:46:51 UTC
Report deadline	10 May 2013, 8:14:02 UTC
Received	28 Mar 2013, 3:24:23 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1264143
Run time	8 days 11 hours 27 min 25 sec
CPU time	7 days 9 hours 44 min 31 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	3.13 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5676, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4432, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5924, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4868, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4672, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4672, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4344, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5800, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4gnuko.pjf4c10 Error converting file to netcdf: dataout/4gnuko.pif4c10 Error converting file to netcdf: dataout/4gnuko.pff4c10 Error converting file to netcdf: dataout/4gnuka.phf4c10 Error converting file to netcdf: dataout/4gnuka.pgf4c10 Error converting file to netcdf: dataout/4gnuka.pef4c10 Error converting file to netcdf: dataout/4gnuka.pdf4c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5040, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5428, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Mar 2013 23:43:04	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	440,640	617,436	1.4012
24 Mar 2013 23:59:56	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	414,720	581,591	1.4024
19 Mar 2013 23:44:50	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	388,800	546,850	1.4065
17 Mar 2013 21:45:02	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	362,880	512,056	1.4111
12 Mar 2013 22:52:21	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	336,960	476,483	1.4141
10 Mar 2013 23:24:49	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	311,040	437,880	1.4078
07 Mar 2013 00:12:08	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	285,120	401,482	1.4081
05 Mar 2013 01:18:45	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	259,200	366,791	1.4151
01 Mar 2013 03:16:10	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	233,280	332,064	1.4235
27 Feb 2013 03:50:42	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	207,360	297,633	1.4353
25 Feb 2013 03:58:05	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	181,440	263,256	1.4509
21 Feb 2013 22:19:36	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	155,520	227,811	1.4648
20 Feb 2013 01:16:34	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	129,600	195,332	1.5072
18 Feb 2013 18:04:26	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	103,680	156,730	1.5117
18 Feb 2013 03:37:09	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	77,760	108,907	1.4006
14 Feb 2013 02:59:29	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	51,840	68,973	1.3305
11 Feb 2013 22:16:54	1264143	15597668	hadcm3n_4gnu_1940_40_008310379_0	25,920	34,202	1.3195