Task 15418511

Name	hadcm3n_zlc2_1880_40_008242597_1
Workunit	8397721
Created	29 Oct 2012, 21:00:39 UTC
Sent	29 Oct 2012, 21:00:46 UTC
Report deadline	29 Jan 2013, 4:27:57 UTC
Received	5 Dec 2012, 14:23:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	998662
Run time	24 days 8 hours 50 min 21 sec
CPU time	22 days 19 hours 2 min 32 sec
Validate state	Invalid
Credit	7,153.92
Device peak FLOPS	1.41 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.36</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 11:17:48 (628): No heartbeat from core client for 30 sec - exiting 11:17:49 (628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:18:42 (4216): No heartbeat from core client for 30 sec - exiting 02:18:43 (4216): No heartbeat from core client for 30 sec - exiting 02:18:44 (4216): No heartbeat from core client for 30 sec - exiting 02:18:45 (4216): No heartbeat from core client for 30 sec - exiting 02:18:46 (4216): No heartbeat from core client for 30 sec - exiting 02:18:48 (4216): No heartbeat from core client for 30 sec - exiting 02:18:49 (4216): No heartbeat from core client for 30 sec - exiting 02:18:50 (4216): No heartbeat from core client for 30 sec - exiting 02:18:51 (4216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:52 (4216): No heartbeat from core client for 30 sec - exiting BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zlc2ko.pj88c10 Error converting file to netcdf: dataout/zlc2ko.pi88c10 Error converting file to netcdf: dataout/zlc2ko.pf88c10 Error converting file to netcdf: dataout/zlc2ka.ph88c10 Error converting file to netcdf: dataout/zlc2ka.pg88c10 Error converting file to netcdf: dataout/zlc2ka.pe88c10 Error converting file to netcdf: dataout/zlc2ka.pd88c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:58:11 (2340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on zlc2ko.daa0c20 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2684, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2684, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Dec 2012 10:21:34	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	596,160	1,929,152	3.2360
30 Nov 2012 11:02:23	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	570,240	1,848,839	3.2422
27 Nov 2012 12:49:28	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	544,320	1,768,137	3.2483
26 Nov 2012 12:31:27	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	518,400	1,683,381	3.2473
24 Nov 2012 20:01:13	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	492,480	1,597,462	3.2437
23 Nov 2012 04:24:39	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	466,560	1,508,989	3.2343
22 Nov 2012 04:26:48	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	440,640	1,425,184	3.2344
21 Nov 2012 04:28:02	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	414,720	1,341,375	3.2344
20 Nov 2012 00:40:07	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	388,800	1,255,071	3.2281
18 Nov 2012 12:35:18	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	362,880	1,171,026	3.2270
16 Nov 2012 18:12:14	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	336,960	1,085,962	3.2228
15 Nov 2012 13:49:55	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	311,040	1,002,436	3.2229
13 Nov 2012 19:41:03	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	285,120	920,353	3.2279
12 Nov 2012 09:07:24	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	259,200	838,809	3.2361
11 Nov 2012 00:41:37	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	233,280	751,368	3.2209
09 Nov 2012 10:45:51	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	207,360	668,745	3.2250
08 Nov 2012 01:33:32	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	181,440	587,912	3.2403
07 Nov 2012 00:56:37	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	155,520	503,452	3.2372
05 Nov 2012 21:43:53	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	129,600	419,226	3.2348
04 Nov 2012 13:31:17	998662	15418511	hadcm3n_zlc2_1880_40_008242597_1	103,680	334,774	3.2289