Task 16837580

Name	hadcm3n_89a1_1980_40_008721444_3
Workunit	8867422
Created	4 Aug 2014, 15:05:02 UTC
Sent	4 Aug 2014, 15:05:11 UTC
Report deadline	3 Nov 2014, 22:32:22 UTC
Received	15 Aug 2014, 5:59:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1322693
Run time	8 days 12 hours 25 min 34 sec
CPU time	8 days 9 hours 4 min 16 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	2.90 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1112, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:02:33 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:04:19 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4288, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4288, iMonCtr=1 Model crash detected, will try to restart... 19:40:28 (4128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4280, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6700, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6700, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2014 20:57:09	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	648,000	700,960	1.0817
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	622,080	672,788	1.0815
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	596,160	645,428	1.0826
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	570,240	618,913	1.0854
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	544,320	591,391	1.0865
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	518,400	563,374	1.0868
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	492,480	536,811	1.0900
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	466,560	508,248	1.0894
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	440,640	479,268	1.0877
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	414,720	451,716	1.0892
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	388,800	424,816	1.0926
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	362,880	397,949	1.0966
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	336,960	370,835	1.1005
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	311,040	344,520	1.1076
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	285,120	318,155	1.1159
14 Aug 2014 19:56:20	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	259,200	291,429	1.1243
08 Aug 2014 04:47:21	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	233,280	264,290	1.1329
07 Aug 2014 22:04:47	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	207,360	236,505	1.1406
07 Aug 2014 04:08:39	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	181,440	207,529	1.1438
06 Aug 2014 19:43:27	1322693	16837580	hadcm3n_89a1_1980_40_008721444_3	155,520	177,313	1.1401