Task 13592595

Name	hadcm3n_ybfg_1900_40_007520085_4
Workunit	7717560
Created	4 Nov 2011, 7:46:57 UTC
Sent	4 Nov 2011, 7:49:42 UTC
Report deadline	3 Feb 2012, 15:16:53 UTC
Received	1 Apr 2012, 20:35:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1178180
Run time	18 days 17 hours 17 min 48 sec
CPU time	15 days 3 hours 30 min 31 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.44 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:19:20 (3760): No heartbeat from core client for 30 sec - exiting 03:19:21 (3760): No heartbeat from core client for 30 sec - exiting 03:19:22 (3760): No heartbeat from core client for 30 sec - exiting 03:19:23 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:11:06 (4776): No heartbeat from core client for 30 sec - exiting 22:11:07 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:11:08 (4776): No heartbeat from core client for 30 sec - exiting 22:11:09 (4776): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:26:38 (5788): No heartbeat from core client for 30 sec - exiting 06:26:39 (5788): No heartbeat from core client for 30 sec - exiting 06:26:40 (5788): No heartbeat from core client for 30 sec - exiting 06:26:41 (5788): No heartbeat from core client for 30 sec - exiting 06:26:42 (5788): No heartbeat from core client for 30 sec - exiting 06:26:43 (5788): No heartbeat from core client for 30 sec - exiting 06:26:44 (5788): No heartbeat from core client for 30 sec - exiting 06:26:45 (5788): No heartbeat from core client for 30 sec - exiting 06:26:46 (5788): No heartbeat from core client for 30 sec - exiting 06:26:47 (5788): No heartbeat from core client for 30 sec - exiting 06:26:48 (5788): No heartbeat from core client for 30 sec - exiting 06:26:49 (5788): No heartbeat from core client for 30 sec - exiting 06:26:50 (5788): No heartbeat from core client for 30 sec - exiting 06:26:51 (5788): No heartbeat from core client for 30 sec - exiting 06:26:52 (5788): No heartbeat from core client for 30 sec - exiting 06:26:53 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:26:54 (5788): No heartbeat from core client for 30 sec - exiting 06:26:55 (5788): No heartbeat from core client for 30 sec - exiting 06:26:56 (5788): No heartbeat from core client for 30 sec - exiting 06:26:57 (5788): No heartbeat from core client for 30 sec - exiting 06:26:58 (5788): No heartbeat from core client for 30 sec - exiting 06:26:59 (5788): No heartbeat from core client for 30 sec - exiting 06:27:00 (5788): No heartbeat from core client for 30 sec - exiting 06:27:01 (5788): No heartbeat from core client for 30 sec - exiting 06:27:02 (5788): No heartbeat from core client for 30 sec - exiting 06:27:03 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5716, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5716, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:48:32 (5648): Can't set up shared mem: -1. Will run in standalone mode. 15:49:25 (3792): Can't set up shared mem: -1. Will run in standalone mode. 14:25:48 (4292): No heartbeat from core client for 30 sec - exiting 14:25:49 (4292): No heartbeat from core client for 30 sec - exiting 14:25:50 (4292): No heartbeat from core client for 30 sec - exiting 14:25:51 (4292): No heartbeat from core client for 30 sec - exiting 14:25:52 (4292): No heartbeat from core client for 30 sec - exiting 14:25:53 (4292): No heartbeat from core client for 30 sec - exiting 14:25:54 (4292): No heartbeat from core client for 30 sec - exiting 14:25:55 (4292): No heartbeat from core client for 30 sec - exiting 14:25:56 (4292): No heartbeat from core client for 30 sec - exiting 14:25:57 (4292): No heartbeat from core client for 30 sec - exiting 14:25:58 (4292): No heartbeat from core client for 30 sec - exiting 14:25:59 (4292): No heartbeat from core client for 30 sec - exiting 14:26:00 (4292): No heartbeat from core client for 30 sec - exiting 14:26:02 (4292): No heartbeat from core client for 30 sec - exiting 14:26:03 (4292): No heartbeat from core client for 30 sec - exiting 14:26:04 (4292): No heartbeat from core client for 30 sec - exiting 14:26:05 (4292): No heartbeat from core client for 30 sec - exiting 14:26:06 (4292): No heartbeat from core client for 30 sec - exiting 14:26:07 (4292): No heartbeat from core client for 30 sec - exiting 14:26:08 (4292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:26:09 (4292): No heartbeat from core client for 30 sec - exiting 14:26:10 (4292): No heartbeat from core client for 30 sec - exiting 14:26:11 (4292): No heartbeat from core client for 30 sec - exiting 14:26:12 (4292): No heartbeat from core client for 30 sec - exiting 14:26:13 (4292): No heartbeat from core client for 30 sec - exiting 14:26:14 (4292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:38:56 (4788): No heartbeat from core client for 30 sec - exiting 19:38:57 (4788): No heartbeat from core client for 30 sec - exiting 19:38:58 (4788): No heartbeat from core client for 30 sec - exiting 19:38:59 (4788): No heartbeat from core client for 30 sec - exiting 19:39:00 (4788): No heartbeat from core client for 30 sec - exiting 19:39:01 (4788): No heartbeat from core client for 30 sec - exiting 19:39:02 (4788): No heartbeat from core client for 30 sec - exiting 19:39:03 (4788): No heartbeat from core client for 30 sec - exiting 19:39:04 (4788): No heartbeat from core client for 30 sec - exiting 19:39:05 (4788): No heartbeat from core client for 30 sec - exiting 19:39:06 (4788): No heartbeat from core client for 30 sec - exiting 19:39:07 (4788): No heartbeat from core client for 30 sec - exiting 19:39:08 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:09 (4788): No heartbeat from core client for 30 sec - exiting 19:39:10 (4788): No heartbeat from core client for 30 sec - exiting 19:39:11 (4788): No heartbeat from core client for 30 sec - exiting 19:39:12 (4788): No heartbeat from core client for 30 sec - exiting 19:39:13 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:24:04 (4888): No heartbeat from core client for 30 sec - exiting 00:24:05 (4888): No heartbeat from core client for 30 sec - exiting 00:24:06 (4888): No heartbeat from core client for 30 sec - exiting 00:24:07 (4888): No heartbeat from core client for 30 sec - exiting 00:24:08 (4888): No heartbeat from core client for 30 sec - exiting 00:24:09 (4888): No heartbeat from core client for 30 sec - exiting 00:24:10 (4888): No heartbeat from core client for 30 sec - exiting 00:24:11 (4888): No heartbeat from core client for 30 sec - exiting 00:24:12 (4888): No heartbeat from core client for 30 sec - exiting 00:24:13 (4888): No heartbeat from core client for 30 sec - exiting 00:24:14 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:24:15 (4888): No heartbeat from core client for 30 sec - exiting 00:24:16 (4888): No heartbeat from core client for 30 sec - exiting 00:24:17 (4888): No heartbeat from core client for 30 sec - exiting 00:24:18 (4888): No heartbeat from core client for 30 sec - exiting 00:24:19 (4888): No heartbeat from core client for 30 sec - exiting 00:24:20 (4888): No heartbeat from core client for 30 sec - exiting 00:24:21 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:45:39 (4368): No heartbeat from core client for 30 sec - exiting 20:45:40 (4368): No heartbeat from core client for 30 sec - exiting 20:45:41 (4368): No heartbeat from core client for 30 sec - exiting 20:45:42 (4368): No heartbeat from core client for 30 sec - exiting 20:45:43 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:08:16 (5320): No heartbeat from core client for 30 sec - exiting 22:08:17 (5320): No heartbeat from core client for 30 sec - exiting 22:08:18 (5320): No heartbeat from core client for 30 sec - exiting 22:08:19 (5320): No heartbeat from core client for 30 sec - exiting 22:08:20 (5320): No heartbeat from core client for 30 sec - exiting 22:08:21 (5320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:08:22 (5320): No heartbeat from core client for 30 sec - exiting 22:08:23 (5320): No heartbeat from core client for 30 sec - exiting 22:08:24 (5320): No heartbeat from core client for 30 sec - exiting 22:08:25 (5320): No heartbeat from core client for 30 sec - exiting 22:08:26 (5320): No heartbeat from core client for 30 sec - exiting 22:08:27 (5320): No heartbeat from core client for 30 sec - exiting 22:08:28 (5320): No heartbeat from core client for 30 sec - exiting 22:08:29 (5320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Apr 2012 20:35:56	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	777,600	1,308,604	1.6829
09 Mar 2012 10:38:17	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	751,680	1,263,283	1.6806
18 Feb 2012 07:34:16	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	725,760	1,217,110	1.6770
04 Feb 2012 08:56:10	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	699,840	1,171,604	1.6741
31 Jan 2012 14:37:47	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	673,920	1,118,504	1.6597
29 Jan 2012 16:00:31	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	648,000	1,074,014	1.6574
24 Jan 2012 15:40:42	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	622,080	1,030,310	1.6562
22 Jan 2012 09:21:14	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	596,160	988,587	1.6583
21 Jan 2012 18:40:53	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	570,240	946,699	1.6602
18 Jan 2012 13:55:19	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	544,320	904,789	1.6622
17 Jan 2012 01:16:36	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	518,400	861,388	1.6616
15 Jan 2012 23:37:40	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	492,480	819,111	1.6632
14 Jan 2012 00:00:09	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	466,560	775,108	1.6613
21 Dec 2011 09:18:04	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	440,640	731,627	1.6604
20 Dec 2011 02:21:21	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	414,720	689,485	1.6625
18 Dec 2011 22:50:19	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	388,800	647,525	1.6654
16 Dec 2011 23:36:27	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	362,880	603,782	1.6639
13 Dec 2011 17:28:52	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	336,960	561,629	1.6668
10 Dec 2011 21:23:36	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	311,040	516,390	1.6602
07 Dec 2011 05:09:43	1178180	13592595	hadcm3n_ybfg_1900_40_007520085_4	285,120	472,965	1.6588