Task 15810375

Name	hadcm3n_o236_1940_40_008379444_0
Workunit	8530303
Created	31 May 2013, 18:12:28 UTC
Sent	18 Jun 2013, 14:02:34 UTC
Report deadline	17 Sep 2013, 21:29:45 UTC
Received	28 Jun 2013, 11:39:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1284796
Run time	5 days 0 hours 25 min 33 sec
CPU time	3 days 19 hours 54 min 44 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	2.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> Устройство не опознает команду. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:47:59 (7800): No heartbeat from core client for 30 sec - exiting 12:48:00 (7800): No heartbeat from core client for 30 sec - exiting 12:48:01 (7800): No heartbeat from core client for 30 sec - exiting 12:48:02 (7800): No heartbeat from core client for 30 sec - exiting 12:48:03 (7800): No heartbeat from core client for 30 sec - exiting 12:48:04 (7800): No heartbeat from core client for 30 sec - exiting 12:48:05 (7800): No heartbeat from core client for 30 sec - exiting 12:48:06 (7800): No heartbeat from core client for 30 sec - exiting 12:48:07 (7800): No heartbeat from core client for 30 sec - exiting 12:48:08 (7800): No heartbeat from core client for 30 sec - exiting 12:48:09 (7800): No heartbeat from core client for 30 sec - exiting 12:48:10 (7800): No heartbeat from core client for 30 sec - exiting 12:48:11 (7800): No heartbeat from core client for 30 sec - exiting 12:48:12 (7800): No heartbeat from core client for 30 sec - exiting 12:48:13 (7800): No heartbeat from core client for 30 sec - exiting 12:48:14 (7800): No heartbeat from core client for 30 sec - exiting 12:48:15 (7800): No heartbeat from core client for 30 sec - exiting 12:48:16 (7800): No heartbeat from core client for 30 sec - exiting 12:48:17 (7800): No heartbeat from core client for 30 sec - exiting 12:48:18 (7800): No heartbeat from core client for 30 sec - exiting 12:48:19 (7800): No heartbeat from core client for 30 sec - exiting 12:48:20 (7800): No heartbeat from core client for 30 sec - exiting 12:48:21 (7800): No heartbeat from core client for 30 sec - exiting 12:48:22 (7800): No heartbeat from core client for 30 sec - exiting 12:48:23 (7800): No heartbeat from core client for 30 sec - exiting 12:48:24 (7800): No heartbeat from core client for 30 sec - exiting 12:48:25 (7800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:48:26 (7800): No heartbeat from core client for 30 sec - exiting 12:48:27 (7800): No heartbeat from core client for 30 sec - exiting 12:48:28 (7800): No heartbeat from core client for 30 sec - exiting 12:48:29 (7800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:57:47 (4048): No heartbeat from core client for 30 sec - exiting 19:57:48 (4048): No heartbeat from core client for 30 sec - exiting 19:57:49 (4048): No heartbeat from core client for 30 sec - exiting 19:57:50 (4048): No heartbeat from core client for 30 sec - exiting 19:57:51 (4048): No heartbeat from core client for 30 sec - exiting 19:57:52 (4048): No heartbeat from core client for 30 sec - exiting 19:57:53 (4048): No heartbeat from core client for 30 sec - exiting 19:57:54 (4048): No heartbeat from core client for 30 sec - exiting 19:57:55 (4048): No heartbeat from core client for 30 sec - exiting 19:57:56 (4048): No heartbeat from core client for 30 sec - exiting 19:57:58 (4048): No heartbeat from core client for 30 sec - exiting 19:57:59 (4048): No heartbeat from core client for 30 sec - exiting 19:58:00 (4048): No heartbeat from core client for 30 sec - exiting 19:58:01 (4048): No heartbeat from core client for 30 sec - exiting 19:58:02 (4048): No heartbeat from core client for 30 sec - exiting 19:58:03 (4048): No heartbeat from core client for 30 sec - exiting 19:58:04 (4048): No heartbeat from core client for 30 sec - exiting 19:58:05 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:58:06 (4048): No heartbeat from core client for 30 sec - exiting 21:13:41 (3424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:42 (3424): No heartbeat from core client for 30 sec - exiting 21:13:43 (3424): No heartbeat from core client for 30 sec - exiting 21:13:44 (3424): No heartbeat from core client for 30 sec - exiting 21:13:45 (3424): No heartbeat from core client for 30 sec - exiting 21:13:46 (3424): No heartbeat from core client for 30 sec - exiting 21:13:47 (3424): No heartbeat from core client for 30 sec - exiting 03:12:37 (7908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:49:09 (9784): No heartbeat from core client for 30 sec - exiting 12:49:10 (9784): No heartbeat from core client for 30 sec - exiting 12:49:11 (9784): No heartbeat from core client for 30 sec - exiting 12:49:12 (9784): No heartbeat from core client for 30 sec - exiting 12:49:13 (9784): No heartbeat from core client for 30 sec - exiting 12:49:14 (9784): No heartbeat from core client for 30 sec - exiting 12:49:15 (9784): No heartbeat from core client for 30 sec - exiting 12:49:16 (9784): No heartbeat from core client for 30 sec - exiting 12:49:17 (9784): No heartbeat from core client for 30 sec - exiting 12:49:18 (9784): No heartbeat from core client for 30 sec - exiting 12:49:19 (9784): No heartbeat from core client for 30 sec - exiting 12:49:20 (9784): No heartbeat from core client for 30 sec - exiting 12:49:21 (9784): No heartbeat from core client for 30 sec - exiting 12:49:22 (9784): No heartbeat from core client for 30 sec - exiting 12:49:23 (9784): No heartbeat from core client for 30 sec - exiting 12:49:24 (9784): No heartbeat from core client for 30 sec - exiting 12:49:25 (9784): No heartbeat from core client for 30 sec - exiting 12:49:26 (9784): No heartbeat from core client for 30 sec - exiting 12:49:27 (9784): No heartbeat from core client for 30 sec - exiting 12:49:28 (9784): No heartbeat from core client for 30 sec - exiting 12:49:29 (9784): No heartbeat from core client for 30 sec - exiting 12:49:30 (9784): No heartbeat from core client for 30 sec - exiting 12:49:31 (9784): No heartbeat from core client for 30 sec - exiting 12:49:32 (9784): No heartbeat from core client for 30 sec - exiting 12:49:33 (9784): No heartbeat from core client for 30 sec - exiting 12:49:34 (9784): No heartbeat from core client for 30 sec - exiting 12:49:35 (9784): No heartbeat from core client for 30 sec - exiting 12:49:36 (9784): No heartbeat from core client for 30 sec - exiting 12:49:37 (9784): No heartbeat from core client for 30 sec - exiting 12:49:38 (9784): No heartbeat from core client for 30 sec - exiting 12:49:39 (9784): No heartbeat from core client for 30 sec - exiting 12:49:40 (9784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:52:01 (9772): No heartbeat from core client for 30 sec - exiting 16:52:02 (9772): No heartbeat from core client for 30 sec - exiting 16:52:03 (9772): No heartbeat from core client for 30 sec - exiting 16:52:04 (9772): No heartbeat from core client for 30 sec - exiting 16:52:05 (9772): No heartbeat from core client for 30 sec - exiting 16:52:06 (9772): No heartbeat from core client for 30 sec - exiting 16:52:07 (9772): No heartbeat from core client for 30 sec - exiting 16:52:08 (9772): No heartbeat from core client for 30 sec - exiting 16:52:09 (9772): No heartbeat from core client for 30 sec - exiting 16:52:10 (9772): No heartbeat from core client for 30 sec - exiting 16:52:11 (9772): No heartbeat from core client for 30 sec - exiting 16:52:12 (9772): No heartbeat from core client for 30 sec - exiting 16:52:13 (9772): No heartbeat from core client for 30 sec - exiting 16:52:14 (9772): No heartbeat from core client for 30 sec - exiting 16:52:15 (9772): No heartbeat from core client for 30 sec - exiting 16:52:16 (9772): No heartbeat from core client for 30 sec - exiting 16:52:17 (9772): No heartbeat from core client for 30 sec - exiting 16:52:18 (9772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:52:19 (9772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:20:00 (9724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:25:31 (8376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:04:03 (7432): No heartbeat from core client for 30 sec - exiting 15:04:04 (7432): No heartbeat from core client for 30 sec - exiting 15:04:05 (7432): No heartbeat from core client for 30 sec - exiting 15:04:06 (7432): No heartbeat from core client for 30 sec - exiting 15:04:07 (7432): No heartbeat from core client for 30 sec - exiting 15:04:08 (7432): No heartbeat from core client for 30 sec - exiting 15:04:09 (7432): No heartbeat from core client for 30 sec - exiting 15:04:10 (7432): No heartbeat from core client for 30 sec - exiting 15:04:11 (7432): No heartbeat from core client for 30 sec - exiting 15:04:12 (7432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:05:26 (12180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:18:47 (3892): No heartbeat from core client for 30 sec - exiting 21:18:49 (3892): No heartbeat from core client for 30 sec - exiting 21:18:50 (3892): No heartbeat from core client for 30 sec - exiting 21:18:51 (3892): No heartbeat from core client for 30 sec - exiting 21:18:52 (3892): No heartbeat from core client for 30 sec - exiting 21:18:53 (3892): No heartbeat from core client for 30 sec - exiting 21:18:54 (3892): No heartbeat from core client for 30 sec - exiting 21:18:55 (3892): No heartbeat from core client for 30 sec - exiting 21:18:56 (3892): No heartbeat from core client for 30 sec - exiting 21:18:57 (3892): No heartbeat from core client for 30 sec - exiting 21:18:58 (3892): No heartbeat from core client for 30 sec - exiting 21:18:59 (3892): No heartbeat from core client for 30 sec - exiting 21:19:00 (3892): No heartbeat from core client for 30 sec - exiting 21:19:01 (3892): No heartbeat from core client for 30 sec - exiting 21:19:02 (3892): No heartbeat from core client for 30 sec - exiting 21:19:03 (3892): No heartbeat from core client for 30 sec - exiting 21:19:04 (3892): No heartbeat from core client for 30 sec - exiting 21:19:05 (3892): No heartbeat from core client for 30 sec - exiting 21:19:06 (3892): No heartbeat from core client for 30 sec - exiting 21:19:07 (3892): No heartbeat from core client for 30 sec - exiting 21:19:08 (3892): No heartbeat from core client for 30 sec - exiting 21:19:09 (3892): No heartbeat from core client for 30 sec - exiting 21:19:10 (3892): No heartbeat from core client for 30 sec - exiting 21:19:11 (3892): No heartbeat from core client for 30 sec - exiting 21:19:12 (3892): No heartbeat from core client for 30 sec - exiting 21:19:13 (3892): No heartbeat from core client for 30 sec - exiting 21:19:14 (3892): No heartbeat from core client for 30 sec - exiting 21:19:15 (3892): No heartbeat from core client for 30 sec - exiting 21:19:16 (3892): No heartbeat from core client for 30 sec - exiting 21:19:17 (3892): No heartbeat from core client for 30 sec - exiting 21:19:18 (3892): No heartbeat from core client for 30 sec - exiting 21:19:19 (3892): No heartbeat from core client for 30 sec - exiting 21:19:20 (3892): No heartbeat from core client for 30 sec - exiting 21:19:21 (3892): No heartbeat from core client for 30 sec - exiting 21:19:22 (3892): No heartbeat from core client for 30 sec - exiting 21:19:23 (3892): No heartbeat from core client for 30 sec - exiting 21:19:24 (3892): No heartbeat from core client for 30 sec - exiting 21:19:25 (3892): No heartbeat from core client for 30 sec - exiting 21:19:26 (3892): No heartbeat from core client for 30 sec - exiting 21:19:27 (3892): No heartbeat from core client for 30 sec - exiting 21:19:28 (3892): No heartbeat from core client for 30 sec - exiting 21:19:29 (3892): No heartbeat from core client for 30 sec - exiting 21:19:30 (3892): No heartbeat from core client for 30 sec - exiting 21:19:31 (3892): No heartbeat from core client for 30 sec - exiting 21:19:32 (3892): No heartbeat from core client for 30 sec - exiting 21:19:33 (3892): No heartbeat from core client for 30 sec - exiting 21:19:34 (3892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:38 (11596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:16:24 (4776): No heartbeat from core client for 30 sec - exiting 01:16:26 (4776): No heartbeat from core client for 30 sec - exiting 01:16:27 (4776): No heartbeat from core client for 30 sec - exiting 01:16:28 (4776): No heartbeat from core client for 30 sec - exiting 01:16:29 (4776): No heartbeat from core client for 30 sec - exiting 01:16:30 (4776): No heartbeat from core client for 30 sec - exiting 01:16:31 (4776): No heartbeat from core client for 30 sec - exiting 01:16:32 (4776): No heartbeat from core client for 30 sec - exiting 01:16:33 (4776): No heartbeat from core client for 30 sec - exiting 01:16:34 (4776): No heartbeat from core client for 30 sec - exiting 01:16:35 (4776): No heartbeat from core client for 30 sec - exiting 01:16:36 (4776): No heartbeat from core client for 30 sec - exiting 01:16:37 (4776): No heartbeat from core client for 30 sec - exiting 01:16:38 (4776): No heartbeat from core client for 30 sec - exiting 01:16:39 (4776): No heartbeat from core client for 30 sec - exiting 01:16:40 (4776): No heartbeat from core client for 30 sec - exiting 01:16:41 (4776): No heartbeat from core client for 30 sec - exiting 01:16:42 (4776): No heartbeat from core client for 30 sec - exiting 01:16:43 (4776): No heartbeat from core client for 30 sec - exiting 01:16:44 (4776): No heartbeat from core client for 30 sec - exiting 01:16:45 (4776): No heartbeat from core client for 30 sec - exiting 01:16:46 (4776): No heartbeat from core client for 30 sec - exiting 01:16:47 (4776): No heartbeat from core client for 30 sec - exiting 01:16:48 (4776): No heartbeat from core client for 30 sec - exiting 01:16:49 (4776): No heartbeat from core client for 30 sec - exiting 01:16:50 (4776): No heartbeat from core client for 30 sec - exiting 01:16:51 (4776): No heartbeat from core client for 30 sec - exiting 01:16:52 (4776): No heartbeat from core client for 30 sec - exiting 01:16:53 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:16:54 (4776): No heartbeat from core client for 30 sec - exiting 01:16:55 (4776): No heartbeat from core client for 30 sec - exiting 01:16:56 (4776): No heartbeat from core client for 30 sec - exiting 01:16:57 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:48:23 (10644): No heartbeat from core client for 30 sec - exiting 00:48:24 (10644): No heartbeat from core client for 30 sec - exiting 00:48:25 (10644): No heartbeat from core client for 30 sec - exiting 00:48:26 (10644): No heartbeat from core client for 30 sec - exiting 00:48:28 (10644): No heartbeat from core client for 30 sec - exiting 00:48:29 (10644): No heartbeat from core client for 30 sec - exiting 00:48:30 (10644): No heartbeat from core client for 30 sec - exiting 00:48:31 (10644): No heartbeat from core client for 30 sec - exiting 00:48:32 (10644): No heartbeat from core client for 30 sec - exiting 00:48:33 (10644): No heartbeat from core client for 30 sec - exiting 00:48:34 (10644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:46:52 (14088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:41:34 (4248): No heartbeat from core client for 30 sec - exiting 10:41:35 (4248): No heartbeat from core client for 30 sec - exiting 10:41:36 (4248): No heartbeat from core client for 30 sec - exiting 10:41:37 (4248): No heartbeat from core client for 30 sec - exiting 10:41:39 (4248): No heartbeat from core client for 30 sec - exiting 10:41:40 (4248): No heartbeat from core client for 30 sec - exiting 10:41:41 (4248): No heartbeat from core client for 30 sec - exiting 10:41:42 (4248): No heartbeat from core client for 30 sec - exiting 10:41:43 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:06:16 (3048): No heartbeat from core client for 30 sec - exiting 11:06:17 (3048): No heartbeat from core client for 30 sec - exiting 11:06:18 (3048): No heartbeat from core client for 30 sec - exiting 11:06:19 (3048): No heartbeat from core client for 30 sec - exiting 11:06:20 (3048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=664, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=664, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=664, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=664, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=664, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Jun 2013 21:57:15	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	207,360	308,589	1.4882
25 Jun 2013 18:41:29	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	181,440	268,472	1.4797
24 Jun 2013 08:44:52	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	155,520	229,112	1.4732
23 Jun 2013 11:17:48	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	129,600	190,847	1.4726
22 Jun 2013 21:31:41	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	103,680	152,030	1.4663
22 Jun 2013 07:10:45	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	77,760	112,573	1.4477
20 Jun 2013 18:25:32	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	51,840	74,843	1.4437
19 Jun 2013 20:35:49	1284796	15810375	hadcm3n_o236_1940_40_008379444_0	25,920	37,261	1.4375