Task 12773057

Name	hadcm3n_o622_1900_40_007203181_2
Workunit	7401461
Created	4 Apr 2011, 14:54:50 UTC
Sent	4 Apr 2011, 14:55:15 UTC
Report deadline	4 Jul 2011, 22:22:26 UTC
Received	19 May 2011, 14:16:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1041826
Run time	14 days 22 hours 28 min 9 sec
CPU time	10 days 19 hours 16 min 2 sec
Validate state	Invalid
Credit	6,842.88
Device peak FLOPS	2.95 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:29:21 (7115): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:29:22 (7115): No heartbeat from core client for 30 sec - exiting 18:29:23 (7115): No heartbeat from core client for 30 sec - exiting 18:29:24 (7115): No heartbeat from core client for 30 sec - exiting 18:29:25 (7115): No heartbeat from core client for 30 sec - exiting 18:29:26 (7115): No heartbeat from core client for 30 sec - exiting 18:29:27 (7115): No heartbeat from core client for 30 sec - exiting 18:29:28 (7115): No heartbeat from core client for 30 sec - exiting 18:29:29 (7115): No heartbeat from core client for 30 sec - exiting 18:29:30 (7115): No heartbeat from core client for 30 sec - exiting 18:29:31 (7115): No heartbeat from core client for 30 sec - exiting 18:29:32 (7115): No heartbeat from core client for 30 sec - exiting 18:29:33 (7115): No heartbeat from core client for 30 sec - exiting 18:29:34 (7115): No heartbeat from core client for 30 sec - exiting 18:29:35 (7115): No heartbeat from core client for 30 sec - exiting 18:29:36 (7115): No heartbeat from core client for 30 sec - exiting 18:29:37 (7115): No heartbeat from core client for 30 sec - exiting 18:29:38 (7115): No heartbeat from core client for 30 sec - exiting 18:29:39 (7115): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:06:12 (30436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:06:13 (30436): No heartbeat from core client for 30 sec - exiting 11:06:14 (30436): No heartbeat from core client for 30 sec - exiting 11:06:15 (30436): No heartbeat from core client for 30 sec - exiting 11:06:16 (30436): No heartbeat from core client for 30 sec - exiting 11:06:17 (30436): No heartbeat from core client for 30 sec - exiting 11:06:18 (30436): No heartbeat from core client for 30 sec - exiting 11:06:19 (30436): No heartbeat from core client for 30 sec - exiting 11:06:20 (30436): No heartbeat from core client for 30 sec - exiting 11:06:21 (30436): No heartbeat from core client for 30 sec - exiting 11:06:22 (30436): No heartbeat from core client for 30 sec - exiting 11:06:23 (30436): No heartbeat from core client for 30 sec - exiting 11:06:24 (30436): No heartbeat from core client for 30 sec - exiting 11:06:25 (30436): No heartbeat from core client for 30 sec - exiting 11:06:26 (30436): No heartbeat from core client for 30 sec - exiting 11:06:27 (30436): No heartbeat from core client for 30 sec - exiting 11:06:28 (30436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:19:25 (41502): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 129135) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=78071, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 129135) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=78071, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 129135) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=78071, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 129135) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=78071, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 129135) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=78071, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 129135) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=78071, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 May 2011 09:35:12	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	570,240	904,563	1.5863
13 May 2011 19:09:06	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	544,320	863,211	1.5859
11 May 2011 01:03:58	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	518,400	824,124	1.5897
07 May 2011 05:50:41	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	492,480	784,101	1.5921
06 May 2011 03:30:51	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	466,560	743,258	1.5931
04 May 2011 17:37:52	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	440,640	702,305	1.5938
03 May 2011 22:40:00	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	414,720	656,977	1.5841
02 May 2011 08:39:33	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	388,800	615,045	1.5819
30 Apr 2011 14:33:25	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	362,880	572,546	1.5778
29 Apr 2011 22:39:24	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	336,960	527,894	1.5666
29 Apr 2011 05:02:20	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	311,040	482,975	1.5528
27 Apr 2011 12:06:15	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	285,120	441,550	1.5486
26 Apr 2011 01:59:01	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	259,200	400,651	1.5457
25 Apr 2011 05:23:10	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	233,280	359,342	1.5404
24 Apr 2011 08:08:01	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	207,360	319,204	1.5394
22 Apr 2011 23:33:48	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	181,440	279,906	1.5427
21 Apr 2011 15:54:30	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	155,520	240,748	1.5480
20 Apr 2011 16:03:00	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	129,600	200,867	1.5499
20 Apr 2011 16:03:00	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	103,680	158,984	1.5334
20 Apr 2011 16:02:59	1041826	12773057	hadcm3n_o622_1900_40_007203181_2	77,760	118,981	1.5301