Task 16068578

Name	hadcm3n_oa1r_1900_40_008468146_1
Workunit	8618985
Created	16 Oct 2013, 19:14:01 UTC
Sent	16 Oct 2013, 19:14:31 UTC
Report deadline	16 Jan 2014, 2:41:42 UTC
Received	14 Dec 2013, 4:43:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1191837
Run time	21 days 8 hours 40 min 34 sec
CPU time	20 days 7 hours 30 min 21 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	1.57 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1864, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2804, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2804, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2804, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2804, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Dec 2013 08:33:16	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	725,760	1,725,512	2.3775
12 Dec 2013 15:55:45	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	699,840	1,666,616	2.3814
11 Dec 2013 22:44:50	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	673,920	1,614,311	2.3954
11 Dec 2013 07:22:41	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	648,000	1,560,213	2.4077
10 Dec 2013 16:09:35	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	622,080	1,506,133	2.4211
09 Dec 2013 14:51:04	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	596,160	1,453,213	2.4376
08 Dec 2013 23:50:36	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	570,240	1,399,947	2.4550
07 Dec 2013 20:59:01	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	544,320	1,346,458	2.4737
07 Dec 2013 06:39:44	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	518,400	1,295,056	2.4982
06 Dec 2013 04:20:54	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	492,480	1,240,312	2.5185
05 Dec 2013 04:31:18	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	466,560	1,187,239	2.5447
04 Dec 2013 10:20:41	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	440,640	1,134,680	2.5751
03 Dec 2013 19:00:36	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	414,720	1,081,235	2.6071
03 Dec 2013 03:22:32	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	388,800	1,023,183	2.6316
02 Dec 2013 09:09:01	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	362,880	971,353	2.6768
01 Dec 2013 18:25:10	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	336,960	917,218	2.7220
26 Nov 2013 07:04:49	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	311,040	851,042	2.7361
22 Nov 2013 20:28:45	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	285,120	775,452	2.7197
21 Nov 2013 07:52:39	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	259,200	700,426	2.7023
19 Nov 2013 04:38:07	1191837	16068578	hadcm3n_oa1r_1900_40_008468146_1	233,280	624,416	2.6767