Task 15879315

Name	hadcm3n_3ltc_1980_40_008398221_0
Workunit	8549077
Created	3 Jul 2013, 18:00:24 UTC
Sent	4 Jul 2013, 9:41:31 UTC
Report deadline	3 Oct 2013, 17:08:42 UTC
Received	6 Sep 2013, 6:04:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1277029
Run time	45 days 4 hours 36 min 2 sec
CPU time	8 days 15 hours 12 min 58 sec
Validate state	Invalid
Credit	7,464.96
Device peak FLOPS	2.76 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> Устройство не опознает команду. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5400, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5400, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... 11:28:52 (5592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5268, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5696, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5696, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5384, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5268, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=200, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4236, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Aug 2013 13:12:49	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	622,080	734,322	1.1804
17 Aug 2013 10:19:53	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	596,160	696,304	1.1680
15 Aug 2013 09:30:53	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	570,240	658,288	1.1544
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	544,320	619,677	1.1384
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	518,400	793,307	1.5303
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	492,480	755,209	1.5335
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	466,560	717,250	1.5373
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	440,640	679,070	1.5411
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	414,720	640,988	1.5456
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	388,800	603,099	1.5512
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	362,880	565,455	1.5582
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	336,960	526,152	1.5615
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	311,040	486,987	1.5657
14 Aug 2013 18:05:28	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	285,120	448,415	1.5727
14 Aug 2013 18:05:27	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	259,200	409,744	1.5808
14 Aug 2013 18:05:27	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	233,280	371,218	1.5913
14 Aug 2013 18:05:27	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	207,360	332,029	1.6012
30 Jul 2013 10:16:02	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	181,440	290,212	1.5995
25 Jul 2013 15:18:00	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	155,520	248,010	1.5947
23 Jul 2013 22:23:40	1277029	15879315	hadcm3n_3ltc_1980_40_008398221_0	129,600	206,145	1.5906