Task 15510906

Name	hadcm3n_o7ih_2140_40_008269392_3
Workunit	8424516
Created	25 Dec 2012, 9:02:16 UTC
Sent	25 Dec 2012, 10:00:37 UTC
Report deadline	26 Mar 2013, 17:27:48 UTC
Received	7 Feb 2013, 19:27:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1126459
Run time	14 days 9 hours 24 min 37 sec
CPU time	14 days 3 hours 49 min 31 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5672, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12492, iMonCtr=1 Model crash detected, will try to restart... MainError: 03:31:47 PM No files match the supplied pattern. MainError: 03:31:47 PM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5880, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 09:26:17 AM No files match the supplied pattern. MainError: 09:26:17 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 12:34:12 AM No files match the supplied pattern. MainError: 12:34:12 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 04:01:47 PM No files match the supplied pattern. MainError: 04:01:47 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 03:09:02 PM No files match the supplied pattern. MainError: 03:09:02 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6280, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 06:08:21 PM No files match the supplied pattern. MainError: 06:08:21 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 04:15:38 PM No files match the supplied pattern. MainError: 04:15:38 PM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4756, iMonCtr=1 Model crash detected, will try to restart... MainError: 07:19:03 PM No files match the supplied pattern. MainError: 07:19:03 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 05:02:24 PM No files match the supplied pattern. MainError: 05:02:24 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 06:49:05 PM No files match the supplied pattern. MainError: 06:49:05 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: dataout/o7ihka.ph11c10 Error converting file to netcdf: dataout/o7ihka.pg11c10 Error converting file to netcdf: dataout/o7ihka.pe11c10 MainError: 06:19:17 PM No files match the supplied pattern. MainError: 06:19:17 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Feb 2013 18:30:02	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	777,600	1,220,947	1.5701
06 Feb 2013 19:05:57	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	751,680	1,181,303	1.5716
05 Feb 2013 17:06:09	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	725,760	1,140,686	1.5717
04 Feb 2013 19:41:05	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	699,840	1,100,940	1.5731
03 Feb 2013 16:16:03	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	673,920	1,061,546	1.5752
02 Feb 2013 18:11:49	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	648,000	1,020,203	1.5744
29 Jan 2013 15:10:31	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	622,080	979,371	1.5743
28 Jan 2013 16:05:39	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	596,160	939,597	1.5761
26 Jan 2013 12:35:07	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	570,240	900,628	1.5794
25 Jan 2013 10:55:41	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	544,320	859,323	1.5787
23 Jan 2013 15:35:36	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	518,400	819,810	1.5814
22 Jan 2013 11:41:02	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	492,480	784,121	1.5922
21 Jan 2013 14:00:31	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	466,560	744,337	1.5954
20 Jan 2013 13:50:33	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	440,640	703,297	1.5961
19 Jan 2013 14:30:28	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	414,720	662,470	1.5974
18 Jan 2013 08:40:04	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	388,800	620,879	1.5969
16 Jan 2013 16:01:11	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	362,880	581,371	1.6021
15 Jan 2013 12:55:13	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	336,960	540,597	1.6043
13 Jan 2013 14:15:16	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	311,040	500,156	1.6080
13 Jan 2013 03:08:35	1126459	15510906	hadcm3n_o7ih_2140_40_008269392_3	285,120	460,275	1.6143