Task 15560113

Name	hadcm3n_zm4h_1880_40_008248218_4
Workunit	8403342
Created	29 Jan 2013, 0:29:05 UTC
Sent	29 Jan 2013, 0:29:12 UTC
Report deadline	30 Apr 2013, 7:56:23 UTC
Received	15 Feb 2013, 6:16:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1260309
Run time	5 days 23 hours 36 min 11 sec
CPU time	5 days 14 hours 44 min 44 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	2.58 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zm4hko.pj93c10 Error converting file to netcdf: dataout/zm4hko.pi93c10 Error converting file to netcdf: dataout/zm4hko.pf93c10 Error converting file to netcdf: dataout/zm4hka.ph93c10 Error converting file to netcdf: dataout/zm4hka.pg93c10 Error converting file to netcdf: dataout/zm4hka.pe93c10 Error converting file to netcdf: dataout/zm4hka.pd93c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8628, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7464, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4584, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Feb 2013 22:14:44	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	414,720	494,785	1.1931
14 Feb 2013 02:44:25	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	388,800	464,851	1.1956
13 Feb 2013 05:40:43	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	362,880	434,357	1.1970
11 Feb 2013 22:16:55	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	336,960	404,869	1.2015
10 Feb 2013 20:41:48	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	311,040	373,613	1.2012
09 Feb 2013 08:11:40	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	285,120	342,090	1.1998
08 Feb 2013 09:12:12	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	259,200	311,927	1.2034
07 Feb 2013 08:02:38	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	233,280	280,441	1.2022
06 Feb 2013 08:48:10	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	207,360	249,819	1.2048
05 Feb 2013 07:06:32	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	181,440	220,857	1.2172
03 Feb 2013 22:09:57	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	155,520	189,756	1.2201
03 Feb 2013 01:36:00	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	129,600	157,278	1.2136
01 Feb 2013 22:23:06	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	103,680	125,897	1.2143
31 Jan 2013 20:37:51	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	77,760	93,888	1.2074
31 Jan 2013 00:34:13	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	51,840	61,934	1.1947
30 Jan 2013 03:27:28	1260309	15560113	hadcm3n_zm4h_1880_40_008248218_4	25,920	31,157	1.2020