Task 15612484

Name	hadcm3n_3hy6_1940_40_008267740_3
Workunit	8422864
Created	18 Feb 2013, 17:00:21 UTC
Sent	18 Feb 2013, 17:00:28 UTC
Report deadline	21 May 2013, 0:27:39 UTC
Received	9 Mar 2013, 14:29:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1237050
Run time	12 days 7 hours 53 min 40 sec
CPU time	11 days 20 hours 12 min 15 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.66 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 08:59:41 (5900): No heartbeat from core client for 30 sec - exiting 08:59:42 (5900): No heartbeat from core client for 30 sec - exiting 08:59:43 (5900): No heartbeat from core client for 30 sec - exiting 08:59:44 (5900): No heartbeat from core client for 30 sec - exiting 08:59:45 (5900): No heartbeat from core client for 30 sec - exiting 08:59:47 (5900): No heartbeat from core client for 30 sec - exiting 08:59:48 (5900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:59:49 (5900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:32:39 (6240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:59:56 (8916): No heartbeat from core client for 30 sec - exiting 13:59:58 (8916): No heartbeat from core client for 30 sec - exiting 13:59:59 (8916): No heartbeat from core client for 30 sec - exiting 14:00:00 (8916): No heartbeat from core client for 30 sec - exiting 14:00:01 (8916): No heartbeat from core client for 30 sec - exiting 14:00:02 (8916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:14:04 (4648): No heartbeat from core client for 30 sec - exiting 15:14:06 (4648): No heartbeat from core client for 30 sec - exiting 15:14:07 (4648): No heartbeat from core client for 30 sec - exiting 15:14:08 (4648): No heartbeat from core client for 30 sec - exiting 15:14:10 (4648): No heartbeat from core client for 30 sec - exiting 15:14:11 (4648): No heartbeat from core client for 30 sec - exiting 15:14:12 (4648): No heartbeat from core client for 30 sec - exiting 15:14:13 (4648): No heartbeat from core client for 30 sec - exiting 15:14:14 (4648): No heartbeat from core client for 30 sec - exiting 15:14:15 (4648): No heartbeat from core client for 30 sec - exiting 15:14:16 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:52:18 (2504): No heartbeat from core client for 30 sec - exiting 13:52:19 (2504): No heartbeat from core client for 30 sec - exiting 13:52:20 (2504): No heartbeat from core client for 30 sec - exiting 13:52:21 (2504): No heartbeat from core client for 30 sec - exiting 13:52:22 (2504): No heartbeat from core client for 30 sec - exiting 13:52:23 (2504): No heartbeat from core client for 30 sec - exiting 13:52:25 (2504): No heartbeat from core client for 30 sec - exiting 13:52:26 (2504): No heartbeat from core client for 30 sec - exiting 13:52:27 (2504): No heartbeat from core client for 30 sec - exiting 13:52:28 (2504): No heartbeat from core client for 30 sec - exiting 13:52:29 (2504): No heartbeat from core client for 30 sec - exiting 13:52:30 (2504): No heartbeat from core client for 30 sec - exiting 13:52:31 (2504): No heartbeat from core client for 30 sec - exiting 13:52:32 (2504): No heartbeat from core client for 30 sec - exiting 13:52:33 (2504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:52:34 (2504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:16:14 (6484): No heartbeat from core client for 30 sec - exiting 14:16:15 (6484): No heartbeat from core client for 30 sec - exiting 14:16:16 (6484): No heartbeat from core client for 30 sec - exiting 14:16:17 (6484): No heartbeat from core client for 30 sec - exiting 14:16:18 (6484): No heartbeat from core client for 30 sec - exiting 14:16:19 (6484): No heartbeat from core client for 30 sec - exiting 14:16:20 (6484): No heartbeat from core client for 30 sec - exiting 14:16:21 (6484): No heartbeat from core client for 30 sec - exiting 14:16:23 (6484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:43 (6808): No heartbeat from core client for 30 sec - exiting 15:23:45 (6808): No heartbeat from core client for 30 sec - exiting 15:23:46 (6808): No heartbeat from core client for 30 sec - exiting 15:23:47 (6808): No heartbeat from core client for 30 sec - exiting 15:23:48 (6808): No heartbeat from core client for 30 sec - exiting 15:23:49 (6808): No heartbeat from core client for 30 sec - exiting 15:23:50 (6808): No heartbeat from core client for 30 sec - exiting 15:23:52 (6808): No heartbeat from core client for 30 sec - exiting 15:23:53 (6808): No heartbeat from core client for 30 sec - exiting 15:23:54 (6808): No heartbeat from core client for 30 sec - exiting 15:23:55 (6808): No heartbeat from core client for 30 sec - exiting 15:23:56 (6808): No heartbeat from core client for 30 sec - exiting 15:23:57 (6808): No heartbeat from core client for 30 sec - exiting 15:23:58 (6808): No heartbeat from core client for 30 sec - exiting 15:23:59 (6808): No heartbeat from core client for 30 sec - exiting 15:24:00 (6808): No heartbeat from core client for 30 sec - exiting 15:24:01 (6808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1280, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6520, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:51:23 (44664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:04:27 (54064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:16:36 (53236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:45:08 (7076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12568, iMonCtr=1 Model crash detected, will try to restart... 13:49:39 (6132): No heartbeat from core client for 30 sec - exiting 13:49:40 (6132): No heartbeat from core client for 30 sec - exiting 13:49:41 (6132): No heartbeat from core client for 30 sec - exiting 13:49:42 (6132): No heartbeat from core client for 30 sec - exiting 13:49:43 (6132): No heartbeat from core client for 30 sec - exiting 13:49:44 (6132): No heartbeat from core client for 30 sec - exiting 13:49:46 (6132): No heartbeat from core client for 30 sec - exiting 13:49:47 (6132): No heartbeat from core client for 30 sec - exiting 13:49:48 (6132): No heartbeat from core client for 30 sec - exiting 13:49:49 (6132): No heartbeat from core client for 30 sec - exiting 13:49:50 (6132): No heartbeat from core client for 30 sec - exiting 13:49:51 (6132): No heartbeat from core client for 30 sec - exiting 13:49:52 (6132): No heartbeat from core client for 30 sec - exiting 13:49:53 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:54 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:53:45 (11496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Mar 2013 13:30:05	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	518,400	1,023,131	1.9736
08 Mar 2013 22:29:35	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	492,480	971,361	1.9724
08 Mar 2013 07:26:03	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	466,560	924,199	1.9809
07 Mar 2013 15:15:31	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	440,640	871,005	1.9767
06 Mar 2013 22:41:11	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	414,720	821,956	1.9820
06 Mar 2013 07:50:36	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	388,800	773,539	1.9896
05 Mar 2013 16:50:51	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	362,880	720,716	1.9861
04 Mar 2013 10:13:48	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	336,960	678,091	2.0124
01 Mar 2013 00:40:30	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	311,040	629,049	2.0224
28 Feb 2013 08:47:46	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	285,120	579,156	2.0313
27 Feb 2013 16:50:56	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	259,200	523,572	2.0200
26 Feb 2013 11:06:01	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	233,280	482,846	2.0698
25 Feb 2013 19:12:59	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	207,360	429,687	2.0722
25 Feb 2013 03:22:51	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	181,440	380,851	2.0990
24 Feb 2013 11:21:26	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	155,520	324,774	2.0883
23 Feb 2013 19:58:06	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	129,600	267,508	2.0641
22 Feb 2013 00:10:08	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	103,680	213,471	2.0589
21 Feb 2013 02:53:21	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	77,760	160,957	2.0699
20 Feb 2013 06:12:32	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	51,840	108,263	2.0884
19 Feb 2013 12:50:10	1237050	15612484	hadcm3n_3hy6_1940_40_008267740_3	25,920	52,475	2.0245