Task 19038261

Name	hadcm3n_sdq5_1940_40_010020714_1
Workunit	10018785
Created	13 Oct 2015, 21:29:14 UTC
Sent	13 Oct 2015, 21:29:42 UTC
Report deadline	13 Jan 2016, 4:56:53 UTC
Received	5 Nov 2015, 14:38:22 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1336821
Run time	9 days 9 hours 6 min 14 sec
CPU time	8 days 3 hours 38 min 49 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	3.12 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.8.44</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 04:50:17 (7868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:42:49 (6736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:04:50 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:45 (7640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:55:51 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:27:23 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:04 (7032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:02:08 (7080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:17:52 (2928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:26 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:56 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:17:11 (5596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:06:22 (944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:37 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:33:35 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:50:24 (4088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7384, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7384, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7384, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7384, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7384, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7384, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2015 14:24:20	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	544,320	688,292	1.2645
05 Nov 2015 14:23:19	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	518,400	654,990	1.2635
05 Nov 2015 14:22:16	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	492,480	620,320	1.2596
05 Nov 2015 14:20:37	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	466,560	586,595	1.2573
05 Nov 2015 14:19:09	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	440,640	553,731	1.2567
05 Nov 2015 14:17:17	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	414,720	519,782	1.2533
05 Nov 2015 14:16:00	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	388,800	486,737	1.2519
05 Nov 2015 14:13:12	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	362,880	452,310	1.2464
05 Nov 2015 14:09:41	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	336,960	418,196	1.2411
28 Oct 2015 10:11:31	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	311,040	384,536	1.2363
28 Oct 2015 00:49:50	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	285,120	354,189	1.2422
27 Oct 2015 09:46:28	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	259,200	316,025	1.2192
26 Oct 2015 22:04:31	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	233,280	281,489	1.2067
26 Oct 2015 18:21:18	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	207,360	249,581	1.2036
26 Oct 2015 18:21:18	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	181,440	218,762	1.2057
26 Oct 2015 18:21:18	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	155,520	188,668	1.2131
26 Oct 2015 18:21:18	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	129,600	154,872	1.1950
26 Oct 2015 18:21:18	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	103,680	124,118	1.1971
26 Oct 2015 18:21:18	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	77,760	90,862	1.1685
26 Oct 2015 18:21:18	1336821	19038261	hadcm3n_sdq5_1940_40_010020714_1	51,840	60,779	1.1724