Task 15900547

Name	hadcm3n_o1r7_1980_40_008402034_0
Workunit	8552890
Created	22 Jul 2013, 15:14:30 UTC
Sent	22 Jul 2013, 15:20:42 UTC
Report deadline	21 Oct 2013, 22:47:53 UTC
Received	14 Aug 2013, 19:22:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1325244
Run time	11 days 23 hours 12 min 39 sec
CPU time	11 days 21 hours 27 min 53 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	3.06 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.57</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:39:36 (4792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:40:47 (4896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:41:24 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:43:45 (1500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:44:22 (1340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:45:00 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:45:37 (1648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:46:14 (3820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3900, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3900, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3900, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	829,440	1,005,016	1.2117
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	803,520	972,985	1.2109
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	777,600	940,246	1.2092
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	751,680	907,724	1.2076
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	725,760	875,564	1.2064
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	699,840	843,718	1.2056
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	673,920	811,798	1.2046
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	648,000	779,783	1.2034
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	622,080	748,646	1.2035
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	596,160	717,867	1.2042
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	570,240	686,814	1.2044
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	544,320	655,721	1.2047
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	518,400	625,976	1.2075
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	492,480	594,812	1.2078
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	466,560	562,445	1.2055
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	440,640	529,978	1.2027
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	414,720	497,517	1.1996
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	388,800	465,167	1.1964
14 Aug 2013 19:24:11	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	362,880	432,778	1.1926
14 Aug 2013 19:24:10	1287164	15900547	hadcm3n_o1r7_1980_40_008402034_0	336,960	399,131	1.1845