Task 14359451

Name	hadcm3n_o4o9_2020_40_007856969_0
Workunit	8012081
Created	4 Apr 2012, 20:09:28 UTC
Sent	4 Apr 2012, 20:17:20 UTC
Report deadline	5 Jul 2012, 3:44:31 UTC
Received	16 Apr 2012, 19:43:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1169903
Run time	8 days 23 hours 14 min 55 sec
CPU time	8 days 8 hours 55 min 23 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	3.27 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> 00:58:32 (74049): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:19:12 (88018): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:30:28 (91416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:47:23 (27562): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:51:25 (36673): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:10 (36782): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:17 (36886): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:53 (37098): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:15:06 (37314): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:19:28 (37436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:35 (37560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:27:46 (37670): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:31:58 (37816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:36:25 (37922): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:40:52 (38055): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:58 (38185): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:48:35 (38282): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:11 (38428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:07 (38524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:04:54 (38609): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:22:31 (63373): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:38:42 (44065): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:25 (83166): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:37:37 (48979): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:45 (53356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:59:07 (12652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:12:53 (32607): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:13:50 (32656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 135760) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34485, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 135760) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34485, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 135760) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34485, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 135760) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=811, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 135760) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=811, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 135760) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=811, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Apr 2012 11:50:14	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	414,720	690,249	1.6644
12 Apr 2012 22:51:22	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	388,800	647,411	1.6652
12 Apr 2012 10:00:40	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	362,880	604,689	1.6664
11 Apr 2012 21:15:33	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	336,960	562,073	1.6681
11 Apr 2012 08:27:45	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	311,040	519,466	1.6701
10 Apr 2012 19:45:40	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	285,120	476,596	1.6716
10 Apr 2012 07:41:20	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	259,200	433,982	1.6743
09 Apr 2012 18:53:02	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	233,280	391,427	1.6779
09 Apr 2012 06:11:31	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	207,360	349,205	1.6841
08 Apr 2012 17:39:17	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	181,440	306,838	1.6911
08 Apr 2012 04:18:53	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	155,520	263,128	1.6919
07 Apr 2012 15:08:51	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	129,600	219,562	1.6942
07 Apr 2012 02:04:50	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	103,680	176,280	1.7002
06 Apr 2012 12:50:27	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	77,760	133,407	1.7156
06 Apr 2012 01:02:41	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	51,840	90,057	1.7372
05 Apr 2012 11:23:56	1169903	14359451	hadcm3n_o4o9_2020_40_007856969_0	25,920	45,592	1.7590