Task 16280785

Name	hadcm3n_3lb1_1980_40_008401739_4
Workunit	8552595
Created	6 Feb 2014, 7:20:52 UTC
Sent	6 Feb 2014, 7:20:59 UTC
Report deadline	8 May 2014, 14:48:10 UTC
Received	19 Feb 2014, 18:33:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1304682
Run time	6 days 5 hours 52 min 18 sec
CPU time	5 days 16 hours 41 min 8 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.16 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:51:27 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:51:28 (5716): No heartbeat from core client for 30 sec - exiting 18:51:29 (5716): No heartbeat from core client for 30 sec - exiting 18:51:30 (5716): No heartbeat from core client for 30 sec - exiting 18:51:31 (5716): No heartbeat from core client for 30 sec - exiting 18:51:32 (5716): No heartbeat from core client for 30 sec - exiting 18:51:33 (5716): No heartbeat from core client for 30 sec - exiting 18:51:35 (5716): No heartbeat from core client for 30 sec - exiting 18:51:36 (5716): No heartbeat from core client for 30 sec - exiting 18:51:37 (5716): No heartbeat from core client for 30 sec - exiting 18:51:38 (5716): No heartbeat from core client for 30 sec - exiting 18:53:19 (5904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:55:43 (6168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:55:44 (6168): No heartbeat from core client for 30 sec - exiting 18:55:45 (6168): No heartbeat from core client for 30 sec - exiting 18:55:46 (6168): No heartbeat from core client for 30 sec - exiting 18:55:47 (6168): No heartbeat from core client for 30 sec - exiting 18:55:48 (6168): No heartbeat from core client for 30 sec - exiting 18:55:49 (6168): No heartbeat from core client for 30 sec - exiting 18:55:50 (6168): No heartbeat from core client for 30 sec - exiting 18:55:51 (6168): No heartbeat from core client for 30 sec - exiting 18:55:53 (6168): No heartbeat from core client for 30 sec - exiting 18:55:54 (6168): No heartbeat from core client for 30 sec - exiting 18:57:52 (6824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:38:33 (1088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:34 (1088): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Feb 2014 18:34:18	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	259,200	480,198	1.8526
18 Feb 2014 12:03:51	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	233,280	437,490	1.8754
18 Feb 2014 12:00:55	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	207,360	386,304	1.8630
14 Feb 2014 11:51:25	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	181,440	339,242	1.8697
13 Feb 2014 10:37:45	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	155,520	293,658	1.8882
12 Feb 2014 17:49:32	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	129,600	248,668	1.9187
11 Feb 2014 07:55:50	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	103,680	200,666	1.9354
09 Feb 2014 15:22:17	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	77,760	146,911	1.8893
08 Feb 2014 11:16:19	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	51,840	97,349	1.8779
07 Feb 2014 07:35:01	1304682	16280785	hadcm3n_3lb1_1980_40_008401739_4	25,920	47,734	1.8416