Task 13115046

Name	hadcm3n_yhje_1900_40_007355588_0
Workunit	7553018
Created	6 Jul 2011, 14:41:03 UTC
Sent	10 Jul 2011, 6:19:39 UTC
Report deadline	9 Oct 2011, 13:46:50 UTC
Received	8 Oct 2011, 19:07:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	678086
Run time	25 days 0 hours 45 min 52 sec
CPU time	23 days 7 hours 19 min 15 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6812, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:53:04 (3516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:53:06 (3516): No heartbeat from core client for 30 sec - exiting 21:53:55 (2236): No heartbeat from core client for 30 sec - exiting 21:53:56 (2236): No heartbeat from core client for 30 sec - exiting 21:53:57 (2236): No heartbeat from core client for 30 sec - exiting 21:53:59 (2236): No heartbeat from core client for 30 sec - exiting 21:54:00 (2236): No heartbeat from core client for 30 sec - exiting 21:54:01 (2236): No heartbeat from core client for 30 sec - exiting 21:54:02 (2236): No heartbeat from core client for 30 sec - exiting 21:54:03 (2236): No heartbeat from core client for 30 sec - exiting 21:54:04 (2236): No heartbeat from core client for 30 sec - exiting 21:54:05 (2236): No heartbeat from core client for 30 sec - exiting 21:54:06 (2236): No heartbeat from core client for 30 sec - exiting 21:54:07 (2236): No heartbeat from core client for 30 sec - exiting 21:54:08 (2236): No heartbeat from core client for 30 sec - exiting 21:54:09 (2236): No heartbeat from core client for 30 sec - exiting 21:54:11 (2236): No heartbeat from core client for 30 sec - exiting 21:54:12 (2236): No heartbeat from core client for 30 sec - exiting 21:54:13 (2236): No heartbeat from core client for 30 sec - exiting 21:54:14 (2236): No heartbeat from core client for 30 sec - exiting 21:54:15 (2236): No heartbeat from core client for 30 sec - exiting 21:54:16 (2236): No heartbeat from core client for 30 sec - exiting 21:54:17 (2236): No heartbeat from core client for 30 sec - exiting 21:54:18 (2236): No heartbeat from core client for 30 sec - exiting 21:54:19 (2236): No heartbeat from core client for 30 sec - exiting 21:54:20 (2236): No heartbeat from core client for 30 sec - exiting 21:54:21 (2236): No heartbeat from core client for 30 sec - exiting 21:54:23 (2236): No heartbeat from core client for 30 sec - exiting 21:54:24 (2236): No heartbeat from core client for 30 sec - exiting 21:54:25 (2236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:16 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:35:35 (6336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:35:36 (6336): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3552, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:00:41 (2416): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:09:01 (2872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:09:02 (2872): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7936, iMonCtr=1 Model crash detected, will try to restart... 17:11:57 (4316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6636, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:33:45 (4580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:35:52 (5592): No heartbeat from core client for 30 sec - exiting 07:35:53 (5592): No heartbeat from core client for 30 sec - exiting 07:35:54 (5592): No heartbeat from core client for 30 sec - exiting 07:35:55 (5592): No heartbeat from core client for 30 sec - exiting 07:35:56 (5592): No heartbeat from core client for 30 sec - exiting 07:35:57 (5592): No heartbeat from core client for 30 sec - exiting 07:35:58 (5592): No heartbeat from core client for 30 sec - exiting 07:35:59 (5592): No heartbeat from core client for 30 sec - exiting 07:36:00 (5592): No heartbeat from core client for 30 sec - exiting 07:36:01 (5592): No heartbeat from core client for 30 sec - exiting 07:36:02 (5592): No heartbeat from core client for 30 sec - exiting 07:36:03 (5592): No heartbeat from core client for 30 sec - exiting 07:36:04 (5592): No heartbeat from core client for 30 sec - exiting 07:36:05 (5592): No heartbeat from core client for 30 sec - exiting 07:36:06 (5592): No heartbeat from core client for 30 sec - exiting 07:36:07 (5592): No heartbeat from core client for 30 sec - exiting 07:36:08 (5592): No heartbeat from core client for 30 sec - exiting 07:36:09 (5592): No heartbeat from core client for 30 sec - exiting 07:36:10 (5592): No heartbeat from core client for 30 sec - exiting 07:36:11 (5592): No heartbeat from core client for 30 sec - exiting 07:36:12 (5592): No heartbeat from core client for 30 sec - exiting 07:36:13 (5592): No heartbeat from core client for 30 sec - exiting 07:36:14 (5592): No heartbeat from core client for 30 sec - exiting 07:36:15 (5592): No heartbeat from core client for 30 sec - exiting 07:36:16 (5592): No heartbeat from core client for 30 sec - exiting 07:36:17 (5592): No heartbeat from core client for 30 sec - exiting 07:36:18 (5592): No heartbeat from core client for 30 sec - exiting 07:36:19 (5592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:42:15 (4772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2208, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2404, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2396, iMonCtr=1 Model crash detected, will try to restart... 08:55:59 (2936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:09:34 (5972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4568, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5884, iMonCtr=1 Model crash detected, will try to restart... 08:18:22 (1860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1784, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1780, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5344, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Oct 2011 18:02:32	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	1,036,800	2,013,542	1.9421
07 Oct 2011 18:57:56	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	1,010,880	1,966,365	1.9452
06 Oct 2011 20:36:08	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	984,960	1,918,242	1.9475
05 Oct 2011 22:13:13	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	959,040	1,870,970	1.9509
05 Oct 2011 06:27:42	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	933,120	1,821,646	1.9522
04 Oct 2011 06:44:43	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	907,200	1,773,766	1.9552
03 Oct 2011 06:35:17	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	881,280	1,725,054	1.9574
02 Oct 2011 09:11:03	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	855,360	1,677,120	1.9607
01 Oct 2011 09:24:03	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	829,440	1,633,375	1.9693
30 Sep 2011 10:17:36	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	803,520	1,590,025	1.9788
29 Sep 2011 12:39:09	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	777,600	1,546,089	1.9883
28 Sep 2011 09:13:51	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	751,680	1,494,257	1.9879
26 Sep 2011 19:16:57	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	725,760	1,439,636	1.9836
25 Sep 2011 16:20:23	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	699,840	1,384,676	1.9786
24 Sep 2011 09:21:46	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	673,920	1,329,632	1.9730
22 Sep 2011 19:11:41	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	648,000	1,277,421	1.9713
21 Sep 2011 20:18:43	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	622,080	1,232,611	1.9814
21 Sep 2011 07:35:40	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	596,160	1,186,748	1.9907
20 Sep 2011 09:49:04	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	570,240	1,142,200	2.0030
12 Sep 2011 18:57:26	678086	13115046	hadcm3n_yhje_1900_40_007355588_0	544,320	1,095,546	2.0127