Task 17814962

Name	hadcm3n_xaor_1980_40_009465951_3
Workunit	9548185
Created	22 Jan 2015, 12:31:29 UTC
Sent	22 Jan 2015, 14:24:33 UTC
Report deadline	23 Apr 2015, 21:51:44 UTC
Received	6 Feb 2015, 20:00:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1286653
Run time	6 days 9 hours 29 min 41 sec
CPU time	5 days 15 hours 5 min 12 sec
Validate state	Invalid
Credit	3,732.48
Device peak FLOPS	2.97 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:17:45 (8148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:17:46 (8148): No heartbeat from core client for 30 sec - exiting 02:17:47 (8148): No heartbeat from core client for 30 sec - exiting 02:17:48 (8148): No heartbeat from core client for 30 sec - exiting 02:17:49 (8148): No heartbeat from core client for 30 sec - exiting 02:17:50 (8148): No heartbeat from core client for 30 sec - exiting 02:17:51 (8148): No heartbeat from core client for 30 sec - exiting 02:17:52 (8148): No heartbeat from core client for 30 sec - exiting 02:17:53 (8148): No heartbeat from core client for 30 sec - exiting 02:17:54 (8148): No heartbeat from core client for 30 sec - exiting 02:17:55 (8148): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 05:23:58 (25708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:26:22 (18296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:26:35 (18296): No heartbeat from core client for 30 sec - exiting 08:43:01 (23980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:22:36 (7144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:28:33 (18888): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:39:46 (18948): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:50:03 (23220): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 07:50:04 (23220): No heartbeat from core client for 30 sec - exiting 07:50:05 (23220): No heartbeat from core client for 30 sec - exiting 07:50:06 (23220): No heartbeat from core client for 30 sec - exiting 07:50:07 (23220): No heartbeat from core client for 30 sec - exiting 07:50:08 (23220): No heartbeat from core client for 30 sec - exiting 07:50:09 (23220): No heartbeat from core client for 30 sec - exiting 07:50:10 (23220): No heartbeat from core client for 30 sec - exiting 07:50:11 (23220): No heartbeat from core client for 30 sec - exiting 07:50:12 (23220): No heartbeat from core client for 30 sec - exiting 07:50:13 (23220): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8108, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8108, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8108, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8108, iMonCtr=1 Model crash detected, will try to restart... 12:43:08 (10368): No heartbeat from core client for 30 sec - exiting 12:43:09 (10368): No heartbeat from core client for 30 sec - exiting 12:43:10 (10368): No heartbeat from core client for 30 sec - exiting 12:43:11 (10368): No heartbeat from core client for 30 sec - exiting 12:43:12 (10368): No heartbeat from core client for 30 sec - exiting 12:43:13 (10368): No heartbeat from core client for 30 sec - exiting 12:43:14 (10368): No heartbeat from core client for 30 sec - exiting 12:43:15 (10368): No heartbeat from core client for 30 sec - exiting 12:43:16 (10368): No heartbeat from core client for 30 sec - exiting 12:43:17 (10368): No heartbeat from core client for 30 sec - exiting 12:43:18 (10368): No heartbeat from core client for 30 sec - exiting 12:43:19 (10368): No heartbeat from core client for 30 sec - exiting 12:43:20 (10368): No heartbeat from core client for 30 sec - exiting 12:43:21 (10368): No heartbeat from core client for 30 sec - exiting 12:43:22 (10368): No heartbeat from core client for 30 sec - exiting 12:43:23 (10368): No heartbeat from core client for 30 sec - exiting 12:43:24 (10368): No heartbeat from core client for 30 sec - exiting 12:43:25 (10368): No heartbeat from core client for 30 sec - exiting 12:43:26 (10368): No heartbeat from core client for 30 sec - exiting 12:43:27 (10368): No heartbeat from core client for 30 sec - exiting 12:43:28 (10368): No heartbeat from core client for 30 sec - exiting 12:43:29 (10368): No heartbeat from core client for 30 sec - exiting 12:43:30 (10368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7608, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7608, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Feb 2015 21:30:09	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	311,040	475,172	1.5277
02 Feb 2015 09:19:45	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	285,120	435,670	1.5280
31 Jan 2015 12:17:32	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	259,200	395,490	1.5258
30 Jan 2015 05:28:18	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	233,280	356,026	1.5262
29 Jan 2015 11:53:23	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	207,360	315,471	1.5214
29 Jan 2015 00:03:02	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	181,440	275,783	1.5200
28 Jan 2015 12:33:37	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	155,520	236,575	1.5212
27 Jan 2015 23:22:11	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	129,600	197,364	1.5229
27 Jan 2015 10:11:41	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	103,680	157,912	1.5231
26 Jan 2015 19:55:18	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	77,760	117,899	1.5162
23 Jan 2015 14:43:46	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	51,840	77,732	1.4995
23 Jan 2015 03:29:27	1286653	17814962	hadcm3n_xaor_1980_40_009465951_3	25,920	39,889	1.5389