Task 15588453

Name	hadcm3n_4gig_1940_40_008302960_0
Workunit	8454095
Created	6 Feb 2013, 21:02:51 UTC
Sent	6 Feb 2013, 21:02:58 UTC
Report deadline	9 May 2013, 4:30:09 UTC
Received	11 Apr 2013, 18:14:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	968211
Run time	12 days 22 hours 6 min 27 sec
CPU time	12 days 1 hours 30 min 5 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:16:00 (10032): No heartbeat from core client for 30 sec - exiting 18:16:01 (10032): No heartbeat from core client for 30 sec - exiting 18:16:02 (10032): No heartbeat from core client for 30 sec - exiting 18:16:03 (10032): No heartbeat from core client for 30 sec - exiting 18:16:04 (10032): No heartbeat from core client for 30 sec - exiting 18:16:05 (10032): No heartbeat from core client for 30 sec - exiting 18:16:06 (10032): No heartbeat from core client for 30 sec - exiting 18:16:07 (10032): No heartbeat from core client for 30 sec - exiting 18:16:08 (10032): No heartbeat from core client for 30 sec - exiting 18:16:09 (10032): No heartbeat from core client for 30 sec - exiting 18:16:10 (10032): No heartbeat from core client for 30 sec - exiting 18:16:11 (10032): No heartbeat from core client for 30 sec - exiting 18:16:12 (10032): No heartbeat from core client for 30 sec - exiting 18:16:13 (10032): No heartbeat from core client for 30 sec - exiting 18:16:14 (10032): No heartbeat from core client for 30 sec - exiting 18:16:15 (10032): No heartbeat from core client for 30 sec - exiting 18:16:16 (10032): No heartbeat from core client for 30 sec - exiting 18:16:17 (10032): No heartbeat from core client for 30 sec - exiting 18:16:18 (10032): No heartbeat from core client for 30 sec - exiting 18:16:19 (10032): No heartbeat from core client for 30 sec - exiting 18:16:20 (10032): No heartbeat from core client for 30 sec - exiting 18:16:21 (10032): No heartbeat from core client for 30 sec - exiting 18:16:22 (10032): No heartbeat from core client for 30 sec - exiting 18:16:23 (10032): No heartbeat from core client for 30 sec - exiting 18:16:24 (10032): No heartbeat from core client for 30 sec - exiting 18:16:25 (10032): No heartbeat from core client for 30 sec - exiting 18:16:26 (10032): No heartbeat from core client for 30 sec - exiting 18:16:27 (10032): No heartbeat from core client for 30 sec - exiting 18:16:28 (10032): No heartbeat from core client for 30 sec - exiting 18:16:29 (10032): No heartbeat from core client for 30 sec - exiting 18:16:30 (10032): No heartbeat from core client for 30 sec - exiting 18:16:31 (10032): No heartbeat from core client for 30 sec - exiting 18:16:32 (10032): No heartbeat from core client for 30 sec - exiting 18:16:33 (10032): No heartbeat from core client for 30 sec - exiting 18:16:34 (10032): No heartbeat from core client for 30 sec - exiting 18:16:35 (10032): No heartbeat from core client for 30 sec - exiting 18:16:36 (10032): No heartbeat from core client for 30 sec - exiting 18:16:37 (10032): No heartbeat from core client for 30 sec - exiting 18:16:38 (10032): No heartbeat from core client for 30 sec - exiting 18:16:39 (10032): No heartbeat from core client for 30 sec - exiting 18:16:40 (10032): No heartbeat from core client for 30 sec - exiting 18:16:41 (10032): No heartbeat from core client for 30 sec - exiting 18:16:42 (10032): No heartbeat from core client for 30 sec - exiting 18:16:43 (10032): No heartbeat from core client for 30 sec - exiting 18:16:44 (10032): No heartbeat from core client for 30 sec - exiting 18:16:45 (10032): No heartbeat from core client for 30 sec - exiting 18:16:46 (10032): No heartbeat from core client for 30 sec - exiting 18:16:47 (10032): No heartbeat from core client for 30 sec - exiting 18:16:48 (10032): No heartbeat from core client for 30 sec - exiting 18:16:49 (10032): No heartbeat from core client for 30 sec - exiting 18:16:50 (10032): No heartbeat from core client for 30 sec - exiting 18:16:51 (10032): No heartbeat from core client for 30 sec - exiting 18:16:52 (10032): No heartbeat from core client for 30 sec - exiting 18:16:53 (10032): No heartbeat from core client for 30 sec - exiting 18:16:54 (10032): No heartbeat from core client for 30 sec - exiting 18:16:55 (10032): No heartbeat from core client for 30 sec - exiting 18:16:56 (10032): No heartbeat from core client for 30 sec - exiting 18:16:57 (10032): No heartbeat from core client for 30 sec - exiting 18:16:58 (10032): No heartbeat from core client for 30 sec - exiting 18:16:59 (10032): No heartbeat from core client for 30 sec - exiting 18:17:00 (10032): No heartbeat from core client for 30 sec - exiting 18:17:01 (10032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:14:42 (8380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8632, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8632, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6124, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6124, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Apr 2013 02:40:56	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	777,600	1,035,386	1.3315
09 Apr 2013 13:16:39	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	751,680	1,001,125	1.3318
09 Apr 2013 01:27:22	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	725,760	967,231	1.3327
08 Apr 2013 01:29:22	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	699,840	932,737	1.3328
02 Apr 2013 22:01:05	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	673,920	897,647	1.3320
02 Apr 2013 08:18:24	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	648,000	863,763	1.3330
01 Apr 2013 04:12:50	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	622,080	829,378	1.3332
31 Mar 2013 11:09:51	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	596,160	795,268	1.3340
30 Mar 2013 22:11:04	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	570,240	760,509	1.3337
29 Mar 2013 18:42:48	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	544,320	725,264	1.3324
29 Mar 2013 06:24:51	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	518,400	690,213	1.3314
28 Mar 2013 17:37:48	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	492,480	655,356	1.3307
28 Mar 2013 05:27:31	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	466,560	621,063	1.3312
24 Mar 2013 11:07:47	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	440,640	586,701	1.3315
24 Mar 2013 01:29:39	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	414,720	552,410	1.3320
23 Mar 2013 14:40:35	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	388,800	517,325	1.3306
23 Mar 2013 03:41:24	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	362,880	482,664	1.3301
22 Mar 2013 16:35:46	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	336,960	447,871	1.3292
22 Mar 2013 06:13:11	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	311,040	413,982	1.3310
21 Mar 2013 20:28:36	968211	15588453	hadcm3n_4gig_1940_40_008302960_0	285,120	379,845	1.3322