Task 15502985

Name	hadcm3n_o4s4_2140_40_008270061_0
Workunit	8425185
Created	24 Dec 2012, 1:00:31 UTC
Sent	24 Dec 2012, 6:06:51 UTC
Report deadline	25 Mar 2013, 13:34:02 UTC
Received	2 Feb 2013, 6:11:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1079850
Run time	13 days 0 hours 19 min 17 sec
CPU time	12 days 17 hours 44 min 1 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.75 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3456, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3496, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4812, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1444, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... MainError: 04:16:12 AM No files match the supplied pattern. MainError: 04:16:12 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 09:34:46 AM No files match the supplied pattern. MainError: 09:34:46 AM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2416, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3212, iMonCtr=1 Model crash detected, will try to restart... MainError: 10:14:03 AM No files match the supplied pattern. MainError: 10:14:03 AM No files match the supplied pattern. MainError: 03:42:04 AM No files match the supplied pattern. MainError: 03:42:04 AM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1772, iMonCtr=1 Model crash detected, will try to restart... MainError: 03:35:02 AM No files match the supplied pattern. MainError: 03:35:02 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 05:36:59 AM No files match the supplied pattern. MainError: 05:36:59 AM No files match the supplied pattern. MainError: 08:41:08 AM No files match the supplied pattern. MainError: 08:41:08 AM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4036, iMonCtr=1 Model crash detected, will try to restart... MainError: 06:13:51 AM No files match the supplied pattern. MainError: 06:13:51 AM No files match the supplied pattern. MainError: 11:07:57 PM No files match the supplied pattern. MainError: 11:07:57 PM No files match the supplied pattern. MainError: 09:33:43 AM No files match the supplied pattern. MainError: 09:33:43 AM No files match the supplied pattern. Error converting file to netcdf: dataout/o4s4ka.ph11c10 Error converting file to netcdf: dataout/o4s4ka.pg11c10 Error converting file to netcdf: dataout/o4s4ka.pe11c10 MainError: 04:42:38 AM No files match the supplied pattern. MainError: 04:42:38 AM No files match the supplied pattern. BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Feb 2013 06:15:02	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	777,600	1,111,738	1.4297
01 Feb 2013 10:32:10	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	751,680	1,075,453	1.4307
31 Jan 2013 23:44:49	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	725,760	1,038,001	1.4302
31 Jan 2013 06:25:15	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	699,840	1,000,351	1.4294
30 Jan 2013 09:08:11	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	673,920	963,937	1.4303
29 Jan 2013 06:29:10	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	648,000	925,376	1.4280
28 Jan 2013 06:54:12	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	622,080	886,713	1.4254
27 Jan 2013 04:20:09	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	596,160	848,980	1.4241
25 Jan 2013 10:55:41	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	570,240	812,159	1.4242
23 Jan 2013 10:33:31	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	544,320	775,520	1.4248
22 Jan 2013 04:49:46	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	518,400	739,751	1.4270
20 Jan 2013 14:30:39	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	492,480	703,946	1.4294
20 Jan 2013 05:24:04	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	466,560	666,712	1.4290
17 Jan 2013 11:05:01	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	440,640	630,587	1.4311
14 Jan 2013 11:14:25	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	414,720	593,570	1.4313
14 Jan 2013 05:37:59	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	388,800	555,207	1.4280
13 Jan 2013 05:18:55	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	362,880	519,064	1.4304
11 Jan 2013 09:44:26	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	336,960	482,139	1.4308
10 Jan 2013 03:42:48	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	311,040	446,523	1.4356
07 Jan 2013 10:56:28	1079850	15502985	hadcm3n_o4s4_2140_40_008270061_0	285,120	409,496	1.4362