Task 18705206

Name	hadcm3n_satw_1980_40_010021725_1
Workunit	10019796
Created	16 Jul 2015, 10:30:25 UTC
Sent	16 Jul 2015, 12:41:53 UTC
Report deadline	15 Oct 2015, 20:09:04 UTC
Received	30 Jul 2015, 3:55:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1364830
Run time	11 days 13 hours 2 min
CPU time	11 days 1 hours 28 min 22 sec
Validate state	Invalid
Credit	6,842.88
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 16:30:33 (4840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:16 (6364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:40:14 (5236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:28 (6444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:29 (6444): No heartbeat from core client for 30 sec - exiting 20:48:30 (6444): No heartbeat from core client for 30 sec - exiting 20:48:31 (6444): No heartbeat from core client for 30 sec - exiting 20:48:32 (6444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 15:12:27 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:27:21 (7988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:19:48 (4760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:27 (7368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:27:49 (6800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:39 (7496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:30 (5584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:06:07 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:11:13 (7968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:28:48 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:31:05 (6888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:41 (5592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:52 (4224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:35:53 (4224): No heartbeat from core client for 30 sec - exiting 12:03:10 (7200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:45:21 (7264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:46:38 (4760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:09:26 (2308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:32:00 (264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:26:26 (5740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:27:37 (2504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:29:42 (4780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:19:51 (3036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:14:18 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:42:26 (9136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:40:33 (10852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:30:29 (6308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:07:16 (10516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:26:36 (9456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8240, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8240, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8128, iMonCtr=1 Model crash detected, will try to restart... 20:53:50 (3896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:53:51 (3896): No heartbeat from core client for 30 sec - exiting 20:53:52 (3896): No heartbeat from core client for 30 sec - exiting 20:53:53 (3896): No heartbeat from core client for 30 sec - exiting 20:53:54 (3896): No heartbeat from core client for 30 sec - exiting 20:53:56 (3896): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5360, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jul 2015 15:52:20	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	570,240	941,380	1.6508
29 Jul 2015 02:03:44	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	544,320	898,772	1.6512
28 Jul 2015 09:53:35	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	518,400	855,676	1.6506
27 Jul 2015 21:26:02	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	492,480	812,653	1.6501
27 Jul 2015 08:11:50	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	466,560	769,193	1.6486
26 Jul 2015 16:03:05	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	440,640	727,041	1.6500
25 Jul 2015 18:22:44	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	414,720	683,248	1.6475
25 Jul 2015 04:28:19	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	388,800	640,181	1.6466
24 Jul 2015 15:40:15	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	362,880	596,190	1.6429
23 Jul 2015 10:41:23	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	336,960	553,170	1.6416
22 Jul 2015 21:47:20	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	311,040	509,145	1.6369
22 Jul 2015 07:06:57	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	285,120	466,201	1.6351
21 Jul 2015 18:30:43	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	259,200	422,920	1.6316
21 Jul 2015 04:44:38	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	233,280	379,905	1.6285
20 Jul 2015 15:43:10	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	207,360	337,862	1.6293
20 Jul 2015 02:02:40	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	181,440	294,531	1.6233
19 Jul 2015 13:47:42	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	155,520	251,969	1.6202
19 Jul 2015 01:46:43	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	129,600	209,572	1.6171
18 Jul 2015 13:18:08	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	103,680	167,163	1.6123
18 Jul 2015 01:21:31	1364830	18705206	hadcm3n_satw_1980_40_010021725_1	77,760	125,834	1.6182