Task 15843968

Name	hadcm3n_o7no_1940_40_008381235_3
Workunit	8532094
Created	15 Jun 2013, 16:38:40 UTC
Sent	15 Jun 2013, 17:05:12 UTC
Report deadline	15 Sep 2013, 0:32:23 UTC
Received	14 Aug 2013, 20:09:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1228459
Run time	7 days 20 hours 30 min 8 sec
CPU time	7 days 15 hours 6 min 15 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 09:27:03 (7468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:03:32 (4448): No heartbeat from core client for 30 sec - exiting 09:03:33 (4448): No heartbeat from core client for 30 sec - exiting 09:03:34 (4448): No heartbeat from core client for 30 sec - exiting 09:03:35 (4448): No heartbeat from core client for 30 sec - exiting 09:03:36 (4448): No heartbeat from core client for 30 sec - exiting 09:03:37 (4448): No heartbeat from core client for 30 sec - exiting 09:03:38 (4448): No heartbeat from core client for 30 sec - exiting 09:03:39 (4448): No heartbeat from core client for 30 sec - exiting 09:03:40 (4448): No heartbeat from core client for 30 sec - exiting 09:03:41 (4448): No heartbeat from core client for 30 sec - exiting 09:03:42 (4448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 08:34:41 (7800): No heartbeat from core client for 30 sec - exiting 08:34:42 (7800): No heartbeat from core client for 30 sec - exiting 08:34:43 (7800): No heartbeat from core client for 30 sec - exiting 08:34:44 (7800): No heartbeat from core client for 30 sec - exiting 08:34:45 (7800): No heartbeat from core client for 30 sec - exiting 08:34:46 (7800): No heartbeat from core client for 30 sec - exiting 08:34:47 (7800): No heartbeat from core client for 30 sec - exiting 08:34:48 (7800): No heartbeat from core client for 30 sec - exiting 08:34:49 (7800): No heartbeat from core client for 30 sec - exiting 08:34:50 (7800): No heartbeat from core client for 30 sec - exiting 08:34:51 (7800): No heartbeat from core client for 30 sec - exiting 08:34:52 (7800): No heartbeat from core client for 30 sec - exiting 08:34:53 (7800): No heartbeat from core client for 30 sec - exiting 08:34:54 (7800): No heartbeat from core client for 30 sec - exiting 08:34:55 (7800): No heartbeat from core client for 30 sec - exiting 08:34:56 (7800): No heartbeat from core client for 30 sec - exiting 08:34:57 (7800): No heartbeat from core client for 30 sec - exiting 08:34:58 (7800): No heartbeat from core client for 30 sec - exiting 08:34:59 (7800): No heartbeat from core client for 30 sec - exiting 08:35:00 (7800): No heartbeat from core client for 30 sec - exiting 08:35:01 (7800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6272, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	518,400	640,067	1.2347
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	492,480	609,873	1.2384
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	466,560	579,217	1.2415
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	440,640	548,186	1.2441
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	414,720	517,094	1.2469
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	388,800	488,899	1.2575
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	362,880	460,924	1.2702
14 Aug 2013 20:09:50	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	336,960	430,571	1.2778
23 Jul 2013 22:08:10	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	311,040	398,965	1.2827
23 Jul 2013 20:20:45	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	285,120	365,831	1.2831
07 Jul 2013 18:44:57	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	259,200	332,481	1.2827
06 Jul 2013 15:09:12	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	233,280	299,427	1.2836
04 Jul 2013 14:27:20	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	207,360	266,580	1.2856
02 Jul 2013 10:48:19	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	181,440	233,779	1.2885
25 Jun 2013 17:21:35	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	155,520	200,621	1.2900
23 Jun 2013 19:31:11	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	129,600	167,094	1.2893
23 Jun 2013 10:17:22	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	103,680	133,600	1.2886
22 Jun 2013 18:16:52	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	77,760	100,153	1.2880
21 Jun 2013 15:57:39	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	51,840	66,616	1.2850
20 Jun 2013 15:41:10	1228459	15843968	hadcm3n_o7no_1940_40_008381235_3	25,920	33,239	1.2824