Task 15445033

Name	hadcm3n_zn4c_1880_40_008247594_0
Workunit	8402718
Created	21 Nov 2012, 8:37:43 UTC
Sent	21 Nov 2012, 8:37:52 UTC
Report deadline	20 Feb 2013, 16:05:03 UTC
Received	18 Jan 2013, 10:01:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1236871
Run time	30 days 9 hours 42 min 6 sec
CPU time	26 days 9 hours 41 min 51 sec
Validate state	Invalid
Credit	11,197.44
Device peak FLOPS	2.61 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3344, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2460, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3264, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3040, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1012, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=968, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3504, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3400, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zn4cko.pja0c10 Error converting file to netcdf: dataout/zn4cko.pia0c10 Error converting file to netcdf: dataout/zn4cko.pfa0c10 Error converting file to netcdf: dataout/zn4cko.pca0c10 Error converting file to netcdf: dataout/zn4cko.pba0c10 Error converting file to netcdf: dataout/zn4cko.paa0c10 Error converting file to netcdf: dataout/zn4cka.pha0c10 Error converting file to netcdf: dataout/zn4cka.pga0c10 Error converting file to netcdf: dataout/zn4cka.pea0c10 Error converting file to netcdf: dataout/zn4cka.pda0c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4296, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2668, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1012, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4200, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4276, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3192, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Jan 2013 14:50:47	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	933,120	2,269,417	2.4321
14 Jan 2013 20:39:31	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	907,200	2,208,618	2.4345
13 Jan 2013 13:35:10	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	881,280	2,147,725	2.4371
11 Jan 2013 18:50:10	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	855,360	2,085,518	2.4382
10 Jan 2013 11:24:06	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	829,440	2,023,887	2.4401
08 Jan 2013 18:30:18	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	803,520	1,964,251	2.4446
07 Jan 2013 11:51:49	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	777,600	1,903,298	2.4477
05 Jan 2013 18:32:59	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	751,680	1,840,960	2.4491
04 Jan 2013 14:16:50	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	725,760	1,778,085	2.4500
02 Jan 2013 19:34:40	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	699,840	1,715,557	2.4514
01 Jan 2013 11:55:13	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	673,920	1,651,739	2.4509
30 Dec 2012 15:15:56	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	648,000	1,587,149	2.4493
28 Dec 2012 18:21:59	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	622,080	1,521,268	2.4455
27 Dec 2012 11:29:55	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	596,160	1,457,431	2.4447
25 Dec 2012 12:45:15	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	570,240	1,392,806	2.4425
23 Dec 2012 17:41:14	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	544,320	1,330,389	2.4441
22 Dec 2012 09:44:33	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	518,400	1,268,340	2.4466
20 Dec 2012 14:47:16	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	492,480	1,202,748	2.4422
18 Dec 2012 20:32:44	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	466,560	1,137,426	2.4379
17 Dec 2012 12:45:12	1236871	15445033	hadcm3n_zn4c_1880_40_008247594_0	440,640	1,073,930	2.4372