Task 16054291

Name	hadcm3n_o787_1940_40_008382658_2
Workunit	8533517
Created	2 Oct 2013, 22:16:40 UTC
Sent	2 Oct 2013, 22:57:02 UTC
Report deadline	2 Jan 2014, 6:24:13 UTC
Received	22 Oct 2013, 22:44:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1282399
Run time	13 days 8 hours 59 min 25 sec
CPU time	12 days 23 hours 0 min 35 sec
Validate state	Invalid
Credit	6,842.88
Device peak FLOPS	2.66 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:54:03 (5388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:54:58 (7368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:24:31 (7280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:18:17 (7500): No heartbeat from core client for 30 sec - exiting 09:18:18 (7500): No heartbeat from core client for 30 sec - exiting 09:18:19 (7500): No heartbeat from core client for 30 sec - exiting 09:18:20 (7500): No heartbeat from core client for 30 sec - exiting 09:18:21 (7500): No heartbeat from core client for 30 sec - exiting 09:18:22 (7500): No heartbeat from core client for 30 sec - exiting 09:18:23 (7500): No heartbeat from core client for 30 sec - exiting 09:18:24 (7500): No heartbeat from core client for 30 sec - exiting 09:18:25 (7500): No heartbeat from core client for 30 sec - exiting 09:18:26 (7500): No heartbeat from core client for 30 sec - exiting 09:18:27 (7500): No heartbeat from core client for 30 sec - exiting 09:18:28 (7500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:10:47 (7104): No heartbeat from core client for 30 sec - exiting 18:10:48 (7104): No heartbeat from core client for 30 sec - exiting 18:10:49 (7104): No heartbeat from core client for 30 sec - exiting 18:10:50 (7104): No heartbeat from core client for 30 sec - exiting 18:10:51 (7104): No heartbeat from core client for 30 sec - exiting 18:10:52 (7104): No heartbeat from core client for 30 sec - exiting 18:10:53 (7104): No heartbeat from core client for 30 sec - exiting 18:10:54 (7104): No heartbeat from core client for 30 sec - exiting 18:10:55 (7104): No heartbeat from core client for 30 sec - exiting 18:10:56 (7104): No heartbeat from core client for 30 sec - exiting 18:10:57 (7104): No heartbeat from core client for 30 sec - exiting 18:10:58 (7104): No heartbeat from core client for 30 sec - exiting 18:10:59 (7104): No heartbeat from core client for 30 sec - exiting 18:11:00 (7104): No heartbeat from core client for 30 sec - exiting 18:11:01 (7104): No heartbeat from core client for 30 sec - exiting 18:11:02 (7104): No heartbeat from core client for 30 sec - exiting 18:11:03 (7104): No heartbeat from core client for 30 sec - exiting 18:11:04 (7104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:05 (7104): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Oct 2013 18:25:20	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	570,240	1,107,556	1.9423
22 Oct 2013 04:41:40	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	544,320	1,058,255	1.9442
21 Oct 2013 13:06:29	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	518,400	1,009,330	1.9470
20 Oct 2013 19:33:33	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	492,480	960,551	1.9504
20 Oct 2013 04:42:29	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	466,560	909,907	1.9502
19 Oct 2013 14:20:51	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	440,640	859,251	1.9500
19 Oct 2013 00:32:18	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	414,720	810,530	1.9544
18 Oct 2013 10:45:51	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	388,800	761,792	1.9593
17 Oct 2013 20:55:18	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	362,880	713,032	1.9649
17 Oct 2013 07:07:29	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	336,960	664,322	1.9715
16 Oct 2013 16:50:18	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	311,040	613,823	1.9735
16 Oct 2013 02:42:19	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	285,120	561,953	1.9709
14 Oct 2013 23:20:38	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	259,200	510,341	1.9689
13 Oct 2013 10:29:50	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	233,280	459,792	1.9710
12 Oct 2013 18:23:07	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	207,360	408,276	1.9689
12 Oct 2013 03:02:37	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	181,440	355,660	1.9602
07 Oct 2013 07:39:28	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	155,520	303,782	1.9533
06 Oct 2013 15:25:46	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	129,600	250,228	1.9308
05 Oct 2013 23:09:28	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	103,680	196,777	1.8979
05 Oct 2013 08:08:41	1282399	16054291	hadcm3n_o787_1940_40_008382658_2	77,760	147,370	1.8952