Task 15792518

Name	hadcm3n_3j3k_1980_40_008370698_0
Workunit	8521557
Created	21 May 2013, 15:35:06 UTC
Sent	21 May 2013, 15:35:13 UTC
Report deadline	20 Aug 2013, 23:02:24 UTC
Received	13 Jun 2013, 13:37:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	914050
Run time	6 days 14 hours 21 min 13 sec
CPU time	5 days 8 hours 51 min 11 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	1.50 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 17:27:47 (3596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:28:32 (4512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:56 (2528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:40:44 (2452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:40:45 (2452): No heartbeat from core client for 30 sec - exiting 02:04:00 (1664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:18:31 (936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:33:18 (2880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:33:29 (4980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:33:30 (4980): No heartbeat from core client for 30 sec - exiting 05:33:31 (4980): No heartbeat from core client for 30 sec - exiting 05:33:32 (4980): No heartbeat from core client for 30 sec - exiting 07:31:50 (3256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:46 (3680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:47 (3680): No heartbeat from core client for 30 sec - exiting 09:33:48 (3680): No heartbeat from core client for 30 sec - exiting 09:33:49 (3680): No heartbeat from core client for 30 sec - exiting 10:33:53 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:16 (1720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:18 (1720): No heartbeat from core client for 30 sec - exiting 11:59:54 (2980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:01:04 (3568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:03:04 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:47 (4988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:48 (4988): No heartbeat from core client for 30 sec - exiting 15:49:36 (2132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:49:38 (2132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:28:40 (3152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:34:57 (456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:59:54 (5836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:59:56 (5836): No heartbeat from core client for 30 sec - exiting 22:35:07 (6100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:19 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:41:53 (1936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:39:18 (2680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:51:00 (5152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:57:49 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:42:42 (5192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:43:48 (5464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:16:24 (4168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:50:05 (4976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:19:57 (2796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:21:06 (4680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:20:20 (3720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:05:37 (3704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:49:53 (3956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:13:00 (5224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:45:10 (2608): No heartbeat from core client for 30 sec - exiting 19:45:11 (2608): No heartbeat from core client for 30 sec - exiting 19:45:12 (2608): No heartbeat from core client for 30 sec - exiting 19:45:13 (2608): No heartbeat from core client for 30 sec - exiting 19:45:14 (2608): No heartbeat from core client for 30 sec - exiting 19:45:15 (2608): No heartbeat from core client for 30 sec - exiting 19:45:16 (2608): No heartbeat from core client for 30 sec - exiting 19:45:17 (2608): No heartbeat from core client for 30 sec - exiting 19:45:18 (2608): No heartbeat from core client for 30 sec - exiting 19:45:19 (2608): No heartbeat from core client for 30 sec - exiting 19:45:20 (2608): No heartbeat from core client for 30 sec - exiting 19:45:21 (2608): No heartbeat from core client for 30 sec - exiting 19:45:22 (2608): No heartbeat from core client for 30 sec - exiting 19:45:23 (2608): No heartbeat from core client for 30 sec - exiting 19:45:24 (2608): No heartbeat from core client for 30 sec - exiting 19:45:25 (2608): No heartbeat from core client for 30 sec - exiting 19:45:26 (2608): No heartbeat from core client for 30 sec - exiting 19:45:27 (2608): No heartbeat from core client for 30 sec - exiting 19:45:28 (2608): No heartbeat from core client for 30 sec - exiting 19:45:29 (2608): No heartbeat from core client for 30 sec - exiting 19:45:30 (2608): No heartbeat from core client for 30 sec - exiting 19:45:31 (2608): No heartbeat from core client for 30 sec - exiting 19:45:32 (2608): No heartbeat from core client for 30 sec - exiting 19:45:33 (2608): No heartbeat from core client for 30 sec - exiting 19:45:34 (2608): No heartbeat from core client for 30 sec - exiting 19:45:35 (2608): No heartbeat from core client for 30 sec - exiting 19:45:36 (2608): No heartbeat from core client for 30 sec - exiting 19:45:37 (2608): No heartbeat from core client for 30 sec - exiting 19:45:38 (2608): No heartbeat from core client for 30 sec - exiting 19:45:39 (2608): No heartbeat from core client for 30 sec - exiting 19:45:40 (2608): No heartbeat from core client for 30 sec - exiting 19:45:41 (2608): No heartbeat from core client for 30 sec - exiting 19:45:42 (2608): No heartbeat from core client for 30 sec - exiting 19:45:43 (2608): No heartbeat from core client for 30 sec - exiting 19:45:44 (2608): No heartbeat from core client for 30 sec - exiting 19:45:45 (2608): No heartbeat from core client for 30 sec - exiting 19:45:46 (2608): No heartbeat from core client for 30 sec - exiting 19:45:47 (2608): No heartbeat from core client for 30 sec - exiting 19:45:49 (2608): No heartbeat from core client for 30 sec - exiting 19:45:50 (2608): No heartbeat from core client for 30 sec - exiting 19:45:51 (2608): No heartbeat from core client for 30 sec - exiting 19:45:52 (2608): No heartbeat from core client for 30 sec - exiting 19:45:53 (2608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:54 (2608): No heartbeat from core client for 30 sec - exiting 19:45:55 (2608): No heartbeat from core client for 30 sec - exiting 19:45:56 (2608): No heartbeat from core client for 30 sec - exiting 19:45:57 (2608): No heartbeat from core client for 30 sec - exiting 19:45:58 (2608): No heartbeat from core client for 30 sec - exiting 19:45:59 (2608): No heartbeat from core client for 30 sec - exiting 19:46:00 (2608): No heartbeat from core client for 30 sec - exiting 19:46:01 (2608): No heartbeat from core client for 30 sec - exiting 19:46:02 (2608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:45:24 (3732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:54:13 (1976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:54:14 (1976): No heartbeat from core client for 30 sec - exiting 12:57:02 (4752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:43:42 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5188, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5188, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5188, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5188, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4624, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4624, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Jun 2013 09:21:18	914050	15792518	hadcm3n_3j3k_1980_40_008370698_0	155,520	444,351	2.8572
08 Jun 2013 20:59:17	914050	15792518	hadcm3n_3j3k_1980_40_008370698_0	129,600	370,664	2.8601
05 Jun 2013 10:16:00	914050	15792518	hadcm3n_3j3k_1980_40_008370698_0	103,680	298,588	2.8799
01 Jun 2013 22:44:53	914050	15792518	hadcm3n_3j3k_1980_40_008370698_0	77,760	224,033	2.8811
29 May 2013 02:45:54	914050	15792518	hadcm3n_3j3k_1980_40_008370698_0	51,840	149,295	2.8799
23 May 2013 10:23:38	914050	15792518	hadcm3n_3j3k_1980_40_008370698_0	25,920	75,906	2.9285