Task 16025063

Name	hadcm3n_84g8_1980_40_008463724_0
Workunit	8614563
Created	19 Sep 2013, 14:33:46 UTC
Sent	21 Sep 2013, 18:30:49 UTC
Report deadline	22 Dec 2013, 1:58:00 UTC
Received	6 Jan 2014, 21:32:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1122348
Run time	23 days 23 hours 45 min 16 sec
CPU time	23 days 21 hours 17 min 1 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.30 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 16:28:56 (2628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:05:21 (1180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:07:03 (5316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:11:40 (6800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:04:25 (1776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5812, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5812, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4348, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4348, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 04:19:20 (7956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:50:38 (3160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:18:25 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:04 (3064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:35 (8108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:07:21 (8004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:02 (5236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:10:41 (5044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:13:02 (5508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:42:15 (9064): No heartbeat from core client for 30 sec - exiting 22:42:17 (9064): No heartbeat from core client for 30 sec - exiting 22:42:18 (9064): No heartbeat from core client for 30 sec - exiting 22:42:19 (9064): No heartbeat from core client for 30 sec - exiting 22:42:20 (9064): No heartbeat from core client for 30 sec - exiting 22:42:21 (9064): No heartbeat from core client for 30 sec - exiting 22:42:22 (9064): No heartbeat from core client for 30 sec - exiting 22:42:23 (9064): No heartbeat from core client for 30 sec - exiting 22:42:24 (9064): No heartbeat from core client for 30 sec - exiting 22:42:25 (9064): No heartbeat from core client for 30 sec - exiting 22:42:26 (9064): No heartbeat from core client for 30 sec - exiting 22:42:27 (9064): No heartbeat from core client for 30 sec - exiting 22:42:28 (9064): No heartbeat from core client for 30 sec - exiting 22:42:29 (9064): No heartbeat from core client for 30 sec - exiting 22:42:30 (9064): No heartbeat from core client for 30 sec - exiting 22:42:31 (9064): No heartbeat from core client for 30 sec - exiting 22:42:32 (9064): No heartbeat from core client for 30 sec - exiting 22:42:33 (9064): No heartbeat from core client for 30 sec - exiting 22:42:34 (9064): No heartbeat from core client for 30 sec - exiting 22:42:35 (9064): No heartbeat from core client for 30 sec - exiting 22:42:36 (9064): No heartbeat from core client for 30 sec - exiting 22:42:37 (9064): No heartbeat from core client for 30 sec - exiting 22:42:38 (9064): No heartbeat from core client for 30 sec - exiting 22:42:39 (9064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5820, iMonCtr=1 Model crash detected, will try to restart... 03:38:42 (6832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:48:42 (1516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5488, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5624, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4996, iMonCtr=1 Model crash detected, will try to restart... 09:04:53 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:54 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:32:36 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:37:40 (6088): No heartbeat from core client for 30 sec - exiting 00:37:41 (6088): No heartbeat from core client for 30 sec - exiting 00:37:42 (6088): No heartbeat from core client for 30 sec - exiting 00:37:43 (6088): No heartbeat from core client for 30 sec - exiting 00:37:44 (6088): No heartbeat from core client for 30 sec - exiting 00:37:45 (6088): No heartbeat from core client for 30 sec - exiting 00:37:46 (6088): No heartbeat from core client for 30 sec - exiting 00:37:47 (6088): No heartbeat from core client for 30 sec - exiting 00:37:48 (6088): No heartbeat from core client for 30 sec - exiting 00:37:49 (6088): No heartbeat from core client for 30 sec - exiting 00:37:50 (6088): No heartbeat from core client for 30 sec - exiting 00:37:51 (6088): No heartbeat from core client for 30 sec - exiting 00:37:52 (6088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:02:09 (6420): No heartbeat from core client for 30 sec - exiting 04:02:10 (6420): No heartbeat from core client for 30 sec - exiting 04:02:11 (6420): No heartbeat from core client for 30 sec - exiting 04:02:12 (6420): No heartbeat from core client for 30 sec - exiting 04:02:13 (6420): No heartbeat from core client for 30 sec - exiting 04:02:14 (6420): No heartbeat from core client for 30 sec - exiting 04:02:15 (6420): No heartbeat from core client for 30 sec - exiting 04:02:16 (6420): No heartbeat from core client for 30 sec - exiting 04:02:17 (6420): No heartbeat from core client for 30 sec - exiting 04:02:18 (6420): No heartbeat from core client for 30 sec - exiting 04:02:19 (6420): No heartbeat from core client for 30 sec - exiting 04:02:20 (6420): No heartbeat from core client for 30 sec - exiting 04:02:21 (6420): No heartbeat from core client for 30 sec - exiting 04:02:22 (6420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6532, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6408, iMonCtr=1 Model crash detected, will try to restart... 01:10:31 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:24 (3276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:14:17 (7544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:26 (7256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:37:14 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:38:57 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:37 (3440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:42:18 (7152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:33:47 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4824, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6496, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5300, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5984, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4548, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4548, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4548, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5772, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5772, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4912, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6336, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5748, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5748, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Jan 2014 21:34:45	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	1,036,800	2,063,815	1.9906
31 Dec 2013 04:18:57	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	1,010,880	2,018,693	1.9970
24 Dec 2013 04:52:44	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	984,960	1,973,252	2.0034
19 Dec 2013 04:24:09	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	959,040	1,918,654	2.0006
11 Dec 2013 09:48:33	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	933,120	1,864,666	1.9983
10 Dec 2013 01:18:44	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	907,200	1,810,984	1.9962
03 Dec 2013 03:27:33	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	881,280	1,756,198	1.9928
28 Nov 2013 10:58:00	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	855,360	1,702,061	1.9899
22 Nov 2013 12:31:48	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	829,440	1,649,798	1.9891
21 Nov 2013 08:57:50	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	803,520	1,599,266	1.9903
18 Nov 2013 05:28:56	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	777,600	1,549,631	1.9928
16 Nov 2013 13:06:36	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	751,680	1,500,840	1.9966
14 Nov 2013 23:58:53	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	725,760	1,450,012	1.9979
10 Nov 2013 21:23:48	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	699,840	1,403,626	2.0056
10 Nov 2013 05:02:03	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	673,920	1,357,228	2.0139
04 Nov 2013 00:16:18	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	648,000	1,309,751	2.0212
02 Nov 2013 13:55:18	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	622,080	1,261,865	2.0285
31 Oct 2013 03:02:23	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	596,160	1,208,391	2.0270
27 Oct 2013 03:06:11	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	570,240	1,154,105	2.0239
26 Oct 2013 07:51:44	1122348	16025063	hadcm3n_84g8_1980_40_008463724_0	544,320	1,103,900	2.0280