Task 16045985

Name	hadcm3n_ofjr_1900_40_008475274_0
Workunit	8626113
Created	27 Sep 2013, 10:35:51 UTC
Sent	27 Sep 2013, 16:40:19 UTC
Report deadline	28 Dec 2013, 0:07:30 UTC
Received	24 Oct 2013, 13:51:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1251442
Run time	11 days 17 hours 49 min 7 sec
CPU time	8 days 23 hours 19 min 24 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	0.96 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8940, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:30:51 (4708): No heartbeat from core client for 30 sec - exiting 19:30:52 (4708): No heartbeat from core client for 30 sec - exiting 19:30:53 (4708): No heartbeat from core client for 30 sec - exiting 19:30:54 (4708): No heartbeat from core client for 30 sec - exiting 19:30:55 (4708): No heartbeat from core client for 30 sec - exiting 19:30:56 (4708): No heartbeat from core client for 30 sec - exiting 19:30:57 (4708): No heartbeat from core client for 30 sec - exiting 19:30:59 (4708): No heartbeat from core client for 30 sec - exiting 19:31:00 (4708): No heartbeat from core client for 30 sec - exiting 19:31:01 (4708): No heartbeat from core client for 30 sec - exiting 19:31:02 (4708): No heartbeat from core client for 30 sec - exiting 19:31:03 (4708): No heartbeat from core client for 30 sec - exiting 19:31:04 (4708): No heartbeat from core client for 30 sec - exiting 19:31:05 (4708): No heartbeat from core client for 30 sec - exiting 19:31:06 (4708): No heartbeat from core client for 30 sec - exiting 19:31:07 (4708): No heartbeat from core client for 30 sec - exiting 19:31:08 (4708): No heartbeat from core client for 30 sec - exiting 19:31:09 (4708): No heartbeat from core client for 30 sec - exiting 19:31:11 (4708): No heartbeat from core client for 30 sec - exiting 19:31:12 (4708): No heartbeat from core client for 30 sec - exiting 19:31:13 (4708): No heartbeat from core client for 30 sec - exiting 19:31:14 (4708): No heartbeat from core client for 30 sec - exiting 19:31:15 (4708): No heartbeat from core client for 30 sec - exiting 19:31:16 (4708): No heartbeat from core client for 30 sec - exiting 19:31:17 (4708): No heartbeat from core client for 30 sec - exiting 19:31:18 (4708): No heartbeat from core client for 30 sec - exiting 19:31:19 (4708): No heartbeat from core client for 30 sec - exiting 19:31:20 (4708): No heartbeat from core client for 30 sec - exiting 19:31:21 (4708): No heartbeat from core client for 30 sec - exiting 19:31:23 (4708): No heartbeat from core client for 30 sec - exiting 19:31:24 (4708): No heartbeat from core client for 30 sec - exiting 19:31:25 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:26 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2364, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2364, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4976, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7900, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:18:47 (5300): No heartbeat from core client for 30 sec - exiting 19:18:49 (5300): No heartbeat from core client for 30 sec - exiting 19:18:50 (5300): No heartbeat from core client for 30 sec - exiting 19:18:51 (5300): No heartbeat from core client for 30 sec - exiting 19:18:52 (5300): No heartbeat from core client for 30 sec - exiting 19:18:53 (5300): No heartbeat from core client for 30 sec - exiting 19:18:54 (5300): No heartbeat from core client for 30 sec - exiting 19:18:55 (5300): No heartbeat from core client for 30 sec - exiting 19:18:56 (5300): No heartbeat from core client for 30 sec - exiting 19:18:57 (5300): No heartbeat from core client for 30 sec - exiting 19:18:58 (5300): No heartbeat from core client for 30 sec - exiting 19:18:59 (5300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5068, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5024, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6616, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:16:44 (6224): No heartbeat from core client for 30 sec - exiting 08:16:45 (6224): No heartbeat from core client for 30 sec - exiting 08:16:46 (6224): No heartbeat from core client for 30 sec - exiting 08:16:47 (6224): No heartbeat from core client for 30 sec - exiting 08:16:48 (6224): No heartbeat from core client for 30 sec - exiting 08:16:49 (6224): No heartbeat from core client for 30 sec - exiting 08:16:50 (6224): No heartbeat from core client for 30 sec - exiting 08:16:51 (6224): No heartbeat from core client for 30 sec - exiting 08:16:53 (6224): No heartbeat from core client for 30 sec - exiting 08:16:54 (6224): No heartbeat from core client for 30 sec - exiting 08:16:55 (6224): No heartbeat from core client for 30 sec - exiting 08:16:56 (6224): No heartbeat from core client for 30 sec - exiting 08:16:57 (6224): No heartbeat from core client for 30 sec - exiting 08:16:58 (6224): No heartbeat from core client for 30 sec - exiting 08:16:59 (6224): No heartbeat from core client for 30 sec - exiting 08:17:00 (6224): No heartbeat from core client for 30 sec - exiting 08:17:01 (6224): No heartbeat from core client for 30 sec - exiting 08:17:02 (6224): No heartbeat from core client for 30 sec - exiting 08:17:03 (6224): No heartbeat from core client for 30 sec - exiting 08:17:05 (6224): No heartbeat from core client for 30 sec - exiting 08:17:06 (6224): No heartbeat from core client for 30 sec - exiting 08:17:07 (6224): No heartbeat from core client for 30 sec - exiting 08:17:08 (6224): No heartbeat from core client for 30 sec - exiting 08:17:09 (6224): No heartbeat from core client for 30 sec - exiting 08:17:10 (6224): No heartbeat from core client for 30 sec - exiting 08:17:11 (6224): No heartbeat from core client for 30 sec - exiting 08:17:12 (6224): No heartbeat from core client for 30 sec - exiting 08:17:13 (6224): No heartbeat from core client for 30 sec - exiting 08:17:14 (6224): No heartbeat from core client for 30 sec - exiting 08:17:15 (6224): No heartbeat from core client for 30 sec - exiting 08:17:17 (6224): No heartbeat from core client for 30 sec - exiting 08:17:18 (6224): No heartbeat from core client for 30 sec - exiting 08:17:19 (6224): No heartbeat from core client for 30 sec - exiting 08:17:20 (6224): No heartbeat from core client for 30 sec - exiting 08:17:21 (6224): No heartbeat from core client for 30 sec - exiting 08:17:22 (6224): No heartbeat from core client for 30 sec - exiting 08:17:23 (6224): No heartbeat from core client for 30 sec - exiting 08:17:24 (6224): No heartbeat from core client for 30 sec - exiting 08:17:25 (6224): No heartbeat from core client for 30 sec - exiting 08:17:26 (6224): No heartbeat from core client for 30 sec - exiting 08:17:28 (6224): No heartbeat from core client for 30 sec - exiting 08:17:29 (6224): No heartbeat from core client for 30 sec - exiting 08:17:30 (6224): No heartbeat from core client for 30 sec - exiting 08:17:31 (6224): No heartbeat from core client for 30 sec - exiting 08:17:32 (6224): No heartbeat from core client for 30 sec - exiting 08:17:33 (6224): No heartbeat from core client for 30 sec - exiting 08:17:34 (6224): No heartbeat from core client for 30 sec - exiting 08:17:35 (6224): No heartbeat from core client for 30 sec - exiting 08:17:36 (6224): No heartbeat from core client for 30 sec - exiting 08:17:37 (6224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3884, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6220, iMonCtr=1 Model crash detected, will try to restart... 19:06:54 (4668): No heartbeat from core client for 30 sec - exiting 19:06:55 (4668): No heartbeat from core client for 30 sec - exiting 19:06:56 (4668): No heartbeat from core client for 30 sec - exiting 19:06:57 (4668): No heartbeat from core client for 30 sec - exiting 19:06:58 (4668): No heartbeat from core client for 30 sec - exiting 19:06:59 (4668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:53:34 (4028): No heartbeat from core client for 30 sec - exiting 10:53:35 (4028): No heartbeat from core client for 30 sec - exiting 10:53:36 (4028): No heartbeat from core client for 30 sec - exiting 10:53:38 (4028): No heartbeat from core client for 30 sec - exiting 10:53:39 (4028): No heartbeat from core client for 30 sec - exiting 10:53:40 (4028): No heartbeat from core client for 30 sec - exiting 10:53:41 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:57:45 (1604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:09:38 (3924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:35:47 (8084): No heartbeat from core client for 30 sec - exiting 10:35:48 (8084): No heartbeat from core client for 30 sec - exiting 10:35:49 (8084): No heartbeat from core client for 30 sec - exiting 10:35:50 (8084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Oct 2013 21:32:46	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	259,200	775,157	2.9906
20 Oct 2013 15:50:39	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	233,280	696,729	2.9867
17 Oct 2013 18:39:11	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	207,360	618,762	2.9840
15 Oct 2013 03:53:15	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	181,440	539,879	2.9755
12 Oct 2013 15:30:06	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	155,520	457,858	2.9440
10 Oct 2013 00:11:25	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	129,600	376,040	2.9015
08 Oct 2013 02:15:14	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	103,680	301,653	2.9095
04 Oct 2013 23:34:50	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	77,760	231,619	2.9786
02 Oct 2013 21:50:51	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	51,840	155,209	2.9940
30 Sep 2013 19:13:59	1251442	16045985	hadcm3n_ofjr_1900_40_008475274_0	25,920	75,712	2.9210