Task 13547947

Name	hadcm3n_ya2o_1900_40_007521129_1
Workunit	7718604
Created	28 Oct 2011, 13:11:24 UTC
Sent	28 Oct 2011, 13:18:00 UTC
Report deadline	27 Jan 2012, 20:45:11 UTC
Received	15 Nov 2011, 17:06:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	837574
Run time	8 days 23 hours 53 min 11 sec
CPU time	6 days 17 hours 34 min 36 sec
Validate state	Invalid
Credit	3,732.48
Device peak FLOPS	2.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.36</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:45:07 (3840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:08 (3840): No heartbeat from core client for 30 sec - exiting 21:45:09 (3840): No heartbeat from core client for 30 sec - exiting 21:45:10 (3840): No heartbeat from core client for 30 sec - exiting 21:45:11 (3840): No heartbeat from core client for 30 sec - exiting 21:45:12 (3840): No heartbeat from core client for 30 sec - exiting 21:45:13 (3840): No heartbeat from core client for 30 sec - exiting 21:45:14 (3840): No heartbeat from core client for 30 sec - exiting 21:45:15 (3840): No heartbeat from core client for 30 sec - exiting 21:45:16 (3840): No heartbeat from core client for 30 sec - exiting 21:45:17 (3840): No heartbeat from core client for 30 sec - exiting 21:45:18 (3840): No heartbeat from core client for 30 sec - exiting 21:45:19 (3840): No heartbeat from core client for 30 sec - exiting 21:45:20 (3840): No heartbeat from core client for 30 sec - exiting 21:45:21 (3840): No heartbeat from core client for 30 sec - exiting 21:45:22 (3840): No heartbeat from core client for 30 sec - exiting 21:45:23 (3840): No heartbeat from core client for 30 sec - exiting 21:45:24 (3840): No heartbeat from core client for 30 sec - exiting 21:45:25 (3840): No heartbeat from core client for 30 sec - exiting 21:45:26 (3840): No heartbeat from core client for 30 sec - exiting 21:45:27 (3840): No heartbeat from core client for 30 sec - exiting 21:45:28 (3840): No heartbeat from core client for 30 sec - exiting 21:45:29 (3840): No heartbeat from core client for 30 sec - exiting 21:45:30 (3840): No heartbeat from core client for 30 sec - exiting 21:45:31 (3840): No heartbeat from core client for 30 sec - exiting 21:45:32 (3840): No heartbeat from core client for 30 sec - exiting 21:45:33 (3840): No heartbeat from core client for 30 sec - exiting 21:45:34 (3840): No heartbeat from core client for 30 sec - exiting 21:45:35 (3840): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 03:37:25 (79448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 10:05:55 (4564): No heartbeat from core client for 30 sec - exiting 10:05:56 (4564): No heartbeat from core client for 30 sec - exiting 10:05:57 (4564): No heartbeat from core client for 30 sec - exiting 10:05:58 (4564): No heartbeat from core client for 30 sec - exiting 10:05:59 (4564): No heartbeat from core client for 30 sec - exiting 10:06:00 (4564): No heartbeat from core client for 30 sec - exiting 10:06:01 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:53:01 (752): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10056, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10056, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10056, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10056, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10056, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10056, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Nov 2011 17:32:04	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	311,040	554,784	1.7836
09 Nov 2011 19:31:53	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	285,120	506,122	1.7751
08 Nov 2011 16:50:24	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	259,200	459,695	1.7735
08 Nov 2011 02:43:39	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	233,280	415,348	1.7805
07 Nov 2011 11:02:42	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	207,360	371,761	1.7928
06 Nov 2011 18:42:29	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	181,440	328,384	1.8099
05 Nov 2011 13:18:22	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	155,520	284,617	1.8301
04 Nov 2011 06:05:06	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	129,600	238,276	1.8385
02 Nov 2011 22:22:23	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	103,680	192,344	1.8552
01 Nov 2011 14:37:05	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	77,760	146,049	1.8782
31 Oct 2011 19:44:09	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	51,840	100,674	1.9420
31 Oct 2011 18:48:55	837574	13547947	hadcm3n_ya2o_1900_40_007521129_1	25,920	48,443	1.8689