Task 13020867

Name	hadcm3n_t14f_1940_40_007313985_2
Workunit	7511415
Created	28 Jun 2011, 14:20:56 UTC
Sent	28 Jun 2011, 14:31:11 UTC
Report deadline	27 Sep 2011, 21:58:22 UTC
Received	29 Jul 2011, 11:50:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1110645
Run time	20 days 8 hours 42 min 14 sec
CPU time	17 days 11 hours 49 min 33 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.8.23</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:24:46 (4636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:48:58 (13892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:49:04 (13892): No heartbeat from core client for 30 sec - exiting 22:49:05 (13892): No heartbeat from core client for 30 sec - exiting 22:49:06 (13892): No heartbeat from core client for 30 sec - exiting 22:49:07 (13892): No heartbeat from core client for 30 sec - exiting 22:49:08 (13892): No heartbeat from core client for 30 sec - exiting 22:49:09 (13892): No heartbeat from core client for 30 sec - exiting 22:49:10 (13892): No heartbeat from core client for 30 sec - exiting 22:49:11 (13892): No heartbeat from core client for 30 sec - exiting 22:49:12 (13892): No heartbeat from core client for 30 sec - exiting 22:49:13 (13892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:50:21 (13132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:00:12 (8580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jul 2011 11:55:24	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	777,600	1,490,869	1.9173
29 Jul 2011 11:55:24	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	751,680	1,445,917	1.9236
29 Jul 2011 11:55:24	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	725,760	1,395,863	1.9233
25 Jul 2011 18:59:30	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	699,840	1,346,660	1.9242
25 Jul 2011 18:17:37	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	673,920	1,297,911	1.9259
25 Jul 2011 17:49:42	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	648,000	1,247,418	1.9250
25 Jul 2011 17:22:06	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	622,080	1,197,360	1.9248
25 Jul 2011 16:19:39	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	596,160	1,146,889	1.9238
25 Jul 2011 15:43:46	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	570,240	1,097,407	1.9245
25 Jul 2011 14:52:48	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	544,320	1,046,319	1.9222
25 Jul 2011 12:36:19	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	518,400	996,208	1.9217
25 Jul 2011 12:36:19	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	492,480	946,445	1.9218
25 Jul 2011 12:36:19	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	466,560	896,295	1.9211
25 Jul 2011 12:36:19	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	440,640	848,266	1.9251
25 Jul 2011 12:36:19	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	414,720	799,906	1.9288
25 Jul 2011 12:36:19	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	388,800	749,320	1.9273
10 Jul 2011 12:08:08	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	362,880	700,103	1.9293
09 Jul 2011 19:36:18	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	336,960	650,169	1.9295
08 Jul 2011 21:35:04	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	311,040	599,755	1.9282
07 Jul 2011 23:55:39	1110645	13020867	hadcm3n_t14f_1940_40_007313985_2	285,120	549,666	1.9278