Task 15864559

Name	hadcm3n_4cor_1980_40_008395883_0
Workunit	8546742
Created	26 Jun 2013, 0:17:18 UTC
Sent	29 Jun 2013, 23:54:10 UTC
Report deadline	29 Sep 2013, 7:21:21 UTC
Received	10 Jul 2013, 11:09:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1284589
Run time	6 days 5 hours 4 min 32 sec
CPU time	6 days 1 hours 25 min 19 sec
Validate state	Invalid
Credit	4,354.56
Device peak FLOPS	2.84 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:14:06 (4168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:14:08 (4168): No heartbeat from core client for 30 sec - exiting 14:14:09 (4168): No heartbeat from core client for 30 sec - exiting 14:14:10 (4168): No heartbeat from core client for 30 sec - exiting 14:14:11 (4168): No heartbeat from core client for 30 sec - exiting 14:14:12 (4168): No heartbeat from core client for 30 sec - exiting 14:14:13 (4168): No heartbeat from core client for 30 sec - exiting 14:14:14 (4168): No heartbeat from core client for 30 sec - exiting 14:14:15 (4168): No heartbeat from core client for 30 sec - exiting 14:14:16 (4168): No heartbeat from core client for 30 sec - exiting 14:14:17 (4168): No heartbeat from core client for 30 sec - exiting 17:49:39 (7924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:49:40 (7924): No heartbeat from core client for 30 sec - exiting 17:49:41 (7924): No heartbeat from core client for 30 sec - exiting 17:49:42 (7924): No heartbeat from core client for 30 sec - exiting 17:49:43 (7924): No heartbeat from core client for 30 sec - exiting 17:49:44 (7924): No heartbeat from core client for 30 sec - exiting 17:49:45 (7924): No heartbeat from core client for 30 sec - exiting 17:49:46 (7924): No heartbeat from core client for 30 sec - exiting 17:49:47 (7924): No heartbeat from core client for 30 sec - exiting 17:49:49 (7924): No heartbeat from core client for 30 sec - exiting 17:49:50 (7924): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3952, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3952, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Jul 2013 05:30:27	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	362,880	493,747	1.3606
06 Jul 2013 18:47:06	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	336,960	457,890	1.3589
06 Jul 2013 08:27:05	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	311,040	422,347	1.3579
06 Jul 2013 05:32:58	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	285,120	386,509	1.3556
06 Jul 2013 05:03:19	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	259,200	351,049	1.3544
06 Jul 2013 04:29:04	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	233,280	315,510	1.3525
04 Jul 2013 16:26:43	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	207,360	280,537	1.3529
04 Jul 2013 14:21:36	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	181,440	245,244	1.3517
03 Jul 2013 17:55:52	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	155,520	209,900	1.3497
02 Jul 2013 20:36:25	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	129,600	178,173	1.3748
02 Jul 2013 12:07:13	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	103,680	142,983	1.3791
02 Jul 2013 11:59:08	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	77,760	106,610	1.3710
02 Jul 2013 11:34:53	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	51,840	71,321	1.3758
02 Jul 2013 11:09:23	1284589	15864559	hadcm3n_4cor_1980_40_008395883_0	25,920	36,939	1.4251