Task 15837040

Name	hadcm3n_4bby_1940_40_008301921_4
Workunit	8453056
Created	10 Jun 2013, 2:13:42 UTC
Sent	10 Jun 2013, 2:23:35 UTC
Report deadline	9 Sep 2013, 9:50:46 UTC
Received	9 Jul 2013, 1:48:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1083664
Run time	7 days 0 hours 41 min 48 sec
CPU time	6 days 13 hours 57 min 29 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	1.41 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:32:33 (56560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:57:13 (90392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:59:30 (105632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:17:56 (5088): No heartbeat from core client for 30 sec - exiting 18:17:57 (5088): No heartbeat from core client for 30 sec - exiting 18:17:58 (5088): No heartbeat from core client for 30 sec - exiting 18:17:59 (5088): No heartbeat from core client for 30 sec - exiting 18:18:00 (5088): No heartbeat from core client for 30 sec - exiting 18:18:02 (5088): No heartbeat from core client for 30 sec - exiting 18:18:03 (5088): No heartbeat from core client for 30 sec - exiting 18:18:04 (5088): No heartbeat from core client for 30 sec - exiting 18:18:05 (5088): No heartbeat from core client for 30 sec - exiting 18:18:06 (5088): No heartbeat from core client for 30 sec - exiting 18:18:07 (5088): No heartbeat from core client for 30 sec - exiting 18:18:08 (5088): No heartbeat from core client for 30 sec - exiting 18:18:09 (5088): No heartbeat from core client for 30 sec - exiting 18:18:10 (5088): No heartbeat from core client for 30 sec - exiting 18:18:11 (5088): No heartbeat from core client for 30 sec - exiting 18:18:12 (5088): No heartbeat from core client for 30 sec - exiting 18:18:14 (5088): No heartbeat from core client for 30 sec - exiting 18:18:15 (5088): No heartbeat from core client for 30 sec - exiting 18:18:16 (5088): No heartbeat from core client for 30 sec - exiting 18:18:17 (5088): No heartbeat from core client for 30 sec - exiting 18:18:18 (5088): No heartbeat from core client for 30 sec - exiting 18:18:19 (5088): No heartbeat from core client for 30 sec - exiting 18:18:20 (5088): No heartbeat from core client for 30 sec - exiting 18:18:21 (5088): No heartbeat from core client for 30 sec - exiting 18:18:22 (5088): No heartbeat from core client for 30 sec - exiting 18:18:23 (5088): No heartbeat from core client for 30 sec - exiting 18:18:24 (5088): No heartbeat from core client for 30 sec - exiting 18:18:26 (5088): No heartbeat from core client for 30 sec - exiting 18:18:27 (5088): No heartbeat from core client for 30 sec - exiting 18:18:28 (5088): No heartbeat from core client for 30 sec - exiting 18:18:29 (5088): No heartbeat from core client for 30 sec - exiting 18:18:30 (5088): No heartbeat from core client for 30 sec - exiting 18:18:31 (5088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:47:50 (3468): No heartbeat from core client for 30 sec - exiting 02:47:51 (3468): No heartbeat from core client for 30 sec - exiting 02:47:52 (3468): No heartbeat from core client for 30 sec - exiting 02:47:53 (3468): No heartbeat from core client for 30 sec - exiting 02:47:54 (3468): No heartbeat from core client for 30 sec - exiting 02:47:55 (3468): No heartbeat from core client for 30 sec - exiting 02:47:56 (3468): No heartbeat from core client for 30 sec - exiting 02:47:57 (3468): No heartbeat from core client for 30 sec - exiting 02:47:58 (3468): No heartbeat from core client for 30 sec - exiting 02:47:59 (3468): No heartbeat from core client for 30 sec - exiting 02:48:00 (3468): No heartbeat from core client for 30 sec - exiting 02:48:01 (3468): No heartbeat from core client for 30 sec - exiting 02:48:02 (3468): No heartbeat from core client for 30 sec - exiting 02:48:03 (3468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:39:07 (2004): No heartbeat from core client for 30 sec - exiting 13:39:08 (2004): No heartbeat from core client for 30 sec - exiting 13:39:09 (2004): No heartbeat from core client for 30 sec - exiting 13:39:10 (2004): No heartbeat from core client for 30 sec - exiting 13:39:11 (2004): No heartbeat from core client for 30 sec - exiting 13:39:12 (2004): No heartbeat from core client for 30 sec - exiting 13:39:13 (2004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:13:15 (5912): No heartbeat from core client for 30 sec - exiting 19:13:16 (5912): No heartbeat from core client for 30 sec - exiting 19:13:17 (5912): No heartbeat from core client for 30 sec - exiting 19:13:18 (5912): No heartbeat from core client for 30 sec - exiting 19:13:19 (5912): No heartbeat from core client for 30 sec - exiting 19:13:20 (5912): No heartbeat from core client for 30 sec - exiting 19:13:21 (5912): No heartbeat from core client for 30 sec - exiting 19:13:22 (5912): No heartbeat from core client for 30 sec - exiting 19:13:23 (5912): No heartbeat from core client for 30 sec - exiting 19:13:25 (5912): No heartbeat from core client for 30 sec - exiting 19:13:26 (5912): No heartbeat from core client for 30 sec - exiting 19:13:27 (5912): No heartbeat from core client for 30 sec - exiting 19:13:28 (5912): No heartbeat from core client for 30 sec - exiting 19:13:29 (5912): No heartbeat from core client for 30 sec - exiting 19:13:30 (5912): No heartbeat from core client for 30 sec - exiting 19:13:31 (5912): No heartbeat from core client for 30 sec - exiting 19:13:32 (5912): No heartbeat from core client for 30 sec - exiting 19:13:33 (5912): No heartbeat from core client for 30 sec - exiting 19:13:34 (5912): No heartbeat from core client for 30 sec - exiting 19:13:35 (5912): No heartbeat from core client for 30 sec - exiting 19:13:37 (5912): No heartbeat from core client for 30 sec - exiting 19:13:38 (5912): No heartbeat from core client for 30 sec - exiting 19:13:39 (5912): No heartbeat from core client for 30 sec - exiting 19:13:40 (5912): No heartbeat from core client for 30 sec - exiting 19:13:41 (5912): No heartbeat from core client for 30 sec - exiting 19:13:42 (5912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:29:19 (7560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:05:56 (9732): No heartbeat from core client for 30 sec - exiting 14:05:57 (9732): No heartbeat from core client for 30 sec - exiting 14:05:58 (9732): No heartbeat from core client for 30 sec - exiting 14:05:59 (9732): No heartbeat from core client for 30 sec - exiting 14:06:00 (9732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:56:57 (4128): No heartbeat from core client for 30 sec - exiting 03:56:58 (4128): No heartbeat from core client for 30 sec - exiting 03:56:59 (4128): No heartbeat from core client for 30 sec - exiting 03:57:00 (4128): No heartbeat from core client for 30 sec - exiting 03:57:01 (4128): No heartbeat from core client for 30 sec - exiting 03:57:02 (4128): No heartbeat from core client for 30 sec - exiting 03:57:03 (4128): No heartbeat from core client for 30 sec - exiting 03:57:04 (4128): No heartbeat from core client for 30 sec - exiting 03:57:05 (4128): No heartbeat from core client for 30 sec - exiting 03:57:06 (4128): No heartbeat from core client for 30 sec - exiting 03:57:07 (4128): No heartbeat from core client for 30 sec - exiting 03:57:08 (4128): No heartbeat from core client for 30 sec - exiting 03:57:09 (4128): No heartbeat from core client for 30 sec - exiting 03:57:10 (4128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:59:25 (3012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:51:10 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:46:16 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:58:08 (16316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:39:00 (21448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:22:45 (23824): No heartbeat from core client for 30 sec - exiting 14:22:46 (23824): No heartbeat from core client for 30 sec - exiting 14:22:47 (23824): No heartbeat from core client for 30 sec - exiting 14:22:48 (23824): No heartbeat from core client for 30 sec - exiting 14:22:49 (23824): No heartbeat from core client for 30 sec - exiting 14:22:50 (23824): No heartbeat from core client for 30 sec - exiting 14:22:51 (23824): No heartbeat from core client for 30 sec - exiting 14:22:52 (23824): No heartbeat from core client for 30 sec - exiting 14:22:53 (23824): No heartbeat from core client for 30 sec - exiting 14:22:54 (23824): No heartbeat from core client for 30 sec - exiting 14:22:55 (23824): No heartbeat from core client for 30 sec - exiting 14:22:56 (23824): No heartbeat from core client for 30 sec - exiting 14:22:57 (23824): No heartbeat from core client for 30 sec - exiting 14:22:58 (23824): No heartbeat from core client for 30 sec - exiting 14:22:59 (23824): No heartbeat from core client for 30 sec - exiting 14:23:00 (23824): No heartbeat from core client for 30 sec - exiting 14:23:01 (23824): No heartbeat from core client for 30 sec - exiting 14:23:02 (23824): No heartbeat from core client for 30 sec - exiting 14:23:03 (23824): No heartbeat from core client for 30 sec - exiting 14:23:04 (23824): No heartbeat from core client for 30 sec - exiting 14:23:05 (23824): No heartbeat from core client for 30 sec - exiting 14:23:06 (23824): No heartbeat from core client for 30 sec - exiting 14:23:07 (23824): No heartbeat from core client for 30 sec - exiting 14:23:08 (23824): No heartbeat from core client for 30 sec - exiting 14:23:09 (23824): No heartbeat from core client for 30 sec - exiting 14:23:10 (23824): No heartbeat from core client for 30 sec - exiting 14:23:11 (23824): No heartbeat from core client for 30 sec - exiting 14:23:12 (23824): No heartbeat from core client for 30 sec - exiting 14:23:13 (23824): No heartbeat from core client for 30 sec - exiting 14:23:14 (23824): No heartbeat from core client for 30 sec - exiting 14:23:15 (23824): No heartbeat from core client for 30 sec - exiting 14:23:16 (23824): No heartbeat from core client for 30 sec - exiting 14:23:17 (23824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:23:18 (23824): No heartbeat from core client for 30 sec - exiting 14:24:03 (21096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:46:02 (24600): No heartbeat from core client for 30 sec - exiting 02:46:03 (24600): No heartbeat from core client for 30 sec - exiting 02:46:04 (24600): No heartbeat from core client for 30 sec - exiting 02:46:05 (24600): No heartbeat from core client for 30 sec - exiting 02:46:06 (24600): No heartbeat from core client for 30 sec - exiting 02:46:07 (24600): No heartbeat from core client for 30 sec - exiting 02:46:08 (24600): No heartbeat from core client for 30 sec - exiting 02:46:09 (24600): No heartbeat from core client for 30 sec - exiting 02:46:10 (24600): No heartbeat from core client for 30 sec - exiting 02:46:11 (24600): No heartbeat from core client for 30 sec - exiting 02:46:12 (24600): No heartbeat from core client for 30 sec - exiting 02:46:13 (24600): No heartbeat from core client for 30 sec - exiting 02:46:14 (24600): No heartbeat from core client for 30 sec - exiting 02:46:15 (24600): No heartbeat from core client for 30 sec - exiting 02:46:16 (24600): No heartbeat from core client for 30 sec - exiting 02:46:17 (24600): No heartbeat from core client for 30 sec - exiting 02:46:18 (24600): No heartbeat from core client for 30 sec - exiting 02:46:19 (24600): No heartbeat from core client for 30 sec - exiting 02:46:20 (24600): No heartbeat from core client for 30 sec - exiting 02:46:21 (24600): No heartbeat from core client for 30 sec - exiting 02:46:22 (24600): No heartbeat from core client for 30 sec - exiting 02:46:23 (24600): No heartbeat from core client for 30 sec - exiting 02:46:24 (24600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:46:25 (24600): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:33:55 (24584): No heartbeat from core client for 30 sec - exiting 12:33:56 (24584): No heartbeat from core client for 30 sec - exiting 12:33:57 (24584): No heartbeat from core client for 30 sec - exiting 12:33:58 (24584): No heartbeat from core client for 30 sec - exiting 12:33:59 (24584): No heartbeat from core client for 30 sec - exiting 12:34:00 (24584): No heartbeat from core client for 30 sec - exiting 12:34:01 (24584): No heartbeat from core client for 30 sec - exiting 12:34:02 (24584): No heartbeat from core client for 30 sec - exiting 12:34:03 (24584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:41:13 (31508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:47:36 (6068): No heartbeat from core client for 30 sec - exiting 22:47:37 (6068): No heartbeat from core client for 30 sec - exiting 22:47:38 (6068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:54:38 (7980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jul 2013 13:02:29	1083664	15837040	hadcm3n_4bby_1940_40_008301921_4	155,520	491,174	3.1583
02 Jul 2013 09:57:56	1083664	15837040	hadcm3n_4bby_1940_40_008301921_4	129,600	409,762	3.1617
25 Jun 2013 22:12:10	1083664	15837040	hadcm3n_4bby_1940_40_008301921_4	103,680	326,234	3.1465
24 Jun 2013 00:15:52	1083664	15837040	hadcm3n_4bby_1940_40_008301921_4	77,760	243,482	3.1312
18 Jun 2013 16:15:46	1083664	15837040	hadcm3n_4bby_1940_40_008301921_4	51,840	163,303	3.1501
16 Jun 2013 02:18:04	1083664	15837040	hadcm3n_4bby_1940_40_008301921_4	25,920	81,014	3.1255