Task 14984712

Name	hadcm3n_o4xq_2100_40_008086024_0
Workunit	8241138
Created	23 Jul 2012, 15:44:03 UTC
Sent	23 Jul 2012, 15:49:55 UTC
Report deadline	22 Oct 2012, 23:17:06 UTC
Received	9 Aug 2012, 20:25:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1215623
Run time	12 days 8 hours 40 min 29 sec
CPU time	11 days 4 hours 37 min 5 sec
Validate state	Invalid
Credit	6,842.88
Device peak FLOPS	2.95 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3708, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:22:36 (3576): No heartbeat from core client for 30 sec - exiting 08:22:37 (3576): No heartbeat from core client for 30 sec - exiting 08:22:38 (3576): No heartbeat from core client for 30 sec - exiting 08:22:39 (3576): No heartbeat from core client for 30 sec - exiting 08:22:40 (3576): No heartbeat from core client for 30 sec - exiting 08:22:41 (3576): No heartbeat from core client for 30 sec - exiting 08:22:42 (3576): No heartbeat from core client for 30 sec - exiting 08:22:44 (3576): No heartbeat from core client for 30 sec - exiting 08:22:45 (3576): No heartbeat from core client for 30 sec - exiting 08:22:46 (3576): No heartbeat from core client for 30 sec - exiting 08:22:47 (3576): No heartbeat from core client for 30 sec - exiting 08:22:48 (3576): No heartbeat from core client for 30 sec - exiting 08:22:49 (3576): No heartbeat from core client for 30 sec - exiting 08:22:50 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:46:12 (3632): No heartbeat from core client for 30 sec - exiting 10:46:15 (3632): No heartbeat from core client for 30 sec - exiting 10:46:16 (3632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3676, iMonCtr=1 Model crash detected, will try to restart... 19:17:56 (3676): No heartbeat from core client for 30 sec - exiting 19:17:57 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3724, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Aug 2012 07:01:17	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	570,240	965,138	1.6925
07 Aug 2012 06:54:06	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	544,320	922,181	1.6942
06 Aug 2012 16:40:14	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	518,400	878,642	1.6949
06 Aug 2012 00:52:37	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	492,480	834,918	1.6953
05 Aug 2012 10:21:48	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	466,560	790,866	1.6951
04 Aug 2012 20:49:16	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	440,640	747,384	1.6961
04 Aug 2012 07:15:20	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	414,720	703,473	1.6963
03 Aug 2012 18:28:49	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	388,800	660,023	1.6976
03 Aug 2012 04:50:52	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	362,880	616,093	1.6978
02 Aug 2012 15:27:52	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	336,960	572,053	1.6977
02 Aug 2012 01:05:39	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	311,040	528,408	1.6988
01 Aug 2012 10:45:48	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	285,120	484,475	1.6992
31 Jul 2012 21:27:41	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	259,200	440,769	1.7005
31 Jul 2012 06:37:07	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	233,280	397,075	1.7021
30 Jul 2012 15:18:05	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	207,360	353,845	1.7064
30 Jul 2012 01:55:29	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	181,440	310,647	1.7121
29 Jul 2012 11:34:48	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	155,520	267,177	1.7180
28 Jul 2012 09:29:15	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	129,600	222,921	1.7201
27 Jul 2012 02:44:00	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	103,680	178,817	1.7247
25 Jul 2012 15:03:32	1215623	14984712	hadcm3n_o4xq_2100_40_008086024_0	77,760	134,467	1.7293