Task 16101758

Name	hadcm3n_82z5_1980_40_008461796_1
Workunit	8612652
Created	2 Dec 2013, 0:07:31 UTC
Sent	2 Dec 2013, 0:07:54 UTC
Report deadline	3 Mar 2014, 7:35:05 UTC
Received	19 Dec 2013, 14:25:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1278973
Run time	6 days 4 hours 31 min 21 sec
CPU time	5 days 14 hours 52 min 49 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	1.89 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> 09:55:34 (5608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:58:40 (6936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1716, iMonCtr=1 Model crash detected, will try to restart... 09:51:45 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:51:46 (3756): No heartbeat from core client for 30 sec - exiting 10:54:50 (9748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:39:45 (7040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:01:07 (4120): No heartbeat from core client for 30 sec - exiting 10:01:08 (4120): No heartbeat from core client for 30 sec - exiting 10:01:09 (4120): No heartbeat from core client for 30 sec - exiting 10:01:10 (4120): No heartbeat from core client for 30 sec - exiting 10:01:11 (4120): No heartbeat from core client for 30 sec - exiting 10:01:12 (4120): No heartbeat from core client for 30 sec - exiting 10:01:13 (4120): No heartbeat from core client for 30 sec - exiting 10:01:14 (4120): No heartbeat from core client for 30 sec - exiting 10:01:15 (4120): No heartbeat from core client for 30 sec - exiting 10:01:16 (4120): No heartbeat from core client for 30 sec - exiting 10:01:17 (4120): No heartbeat from core client for 30 sec - exiting 10:01:18 (4120): No heartbeat from core client for 30 sec - exiting 10:01:19 (4120): No heartbeat from core client for 30 sec - exiting 10:01:20 (4120): No heartbeat from core client for 30 sec - exiting 10:01:21 (4120): No heartbeat from core client for 30 sec - exiting 10:01:22 (4120): No heartbeat from core client for 30 sec - exiting 10:01:23 (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:40:36 (5540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1480, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6192, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5844, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6520, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3396, iMonCtr=1 Model crash detected, will try to restart... 14:49:35 (5388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:58:11 (6892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:43:30 (1180): No heartbeat from core client for 30 sec - exiting 16:43:31 (1180): No heartbeat from core client for 30 sec - exiting 16:43:32 (1180): No heartbeat from core client for 30 sec - exiting 16:43:33 (1180): No heartbeat from core client for 30 sec - exiting 16:43:34 (1180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:35 (1180): No heartbeat from core client for 30 sec - exiting 16:43:36 (1180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 06:54:46 (8044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1264, iMonCtr=1 Model crash detected, will try to restart... 10:01:29 (7620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:01:32 (7620): No heartbeat from core client for 30 sec - exiting 10:01:33 (7620): No heartbeat from core client for 30 sec - exiting 10:01:34 (7620): No heartbeat from core client for 30 sec - exiting 10:01:35 (7620): No heartbeat from core client for 30 sec - exiting 10:01:36 (7620): No heartbeat from core client for 30 sec - exiting 10:01:37 (7620): No heartbeat from core client for 30 sec - exiting 10:01:38 (7620): No heartbeat from core client for 30 sec - exiting 10:01:39 (7620): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 00:07:24 (2460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:27 (2460): No heartbeat from core client for 30 sec - exiting 00:07:28 (2460): No heartbeat from core client for 30 sec - exiting 00:07:29 (2460): No heartbeat from core client for 30 sec - exiting 00:07:30 (2460): No heartbeat from core client for 30 sec - exiting 00:07:31 (2460): No heartbeat from core client for 30 sec - exiting 00:07:32 (2460): No heartbeat from core client for 30 sec - exiting 00:07:33 (2460): No heartbeat from core client for 30 sec - exiting 00:07:34 (2460): No heartbeat from core client for 30 sec - exiting 00:07:35 (2460): No heartbeat from core client for 30 sec - exiting 00:07:36 (2460): No heartbeat from core client for 30 sec - exiting 00:07:37 (2460): No heartbeat from core client for 30 sec - exiting 00:07:39 (2460): No heartbeat from core client for 30 sec - exiting 00:07:40 (2460): No heartbeat from core client for 30 sec - exiting 00:07:41 (2460): No heartbeat from core client for 30 sec - exiting 00:07:42 (2460): No heartbeat from core client for 30 sec - exiting 00:07:43 (2460): No heartbeat from core client for 30 sec - exiting 00:07:44 (2460): No heartbeat from core client for 30 sec - exiting 00:07:45 (2460): No heartbeat from core client for 30 sec - exiting 00:07:46 (2460): No heartbeat from core client for 30 sec - exiting 00:07:47 (2460): No heartbeat from core client for 30 sec - exiting 00:07:48 (2460): No heartbeat from core client for 30 sec - exiting 00:07:49 (2460): No heartbeat from core client for 30 sec - exiting 00:07:51 (2460): No heartbeat from core client for 30 sec - exiting 00:07:52 (2460): No heartbeat from core client for 30 sec - exiting 00:07:53 (2460): No heartbeat from core client for 30 sec - exiting 00:07:54 (2460): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Dec 2013 22:33:34	1278973	16101758	hadcm3n_82z5_1980_40_008461796_1	155,520	459,667	2.9557
14 Dec 2013 00:48:09	1278973	16101758	hadcm3n_82z5_1980_40_008461796_1	129,600	376,701	2.9066
11 Dec 2013 21:54:19	1278973	16101758	hadcm3n_82z5_1980_40_008461796_1	103,680	297,477	2.8692
10 Dec 2013 00:13:19	1278973	16101758	hadcm3n_82z5_1980_40_008461796_1	77,760	211,360	2.7181
05 Dec 2013 21:12:28	1278973	16101758	hadcm3n_82z5_1980_40_008461796_1	51,840	139,190	2.6850
04 Dec 2013 14:46:00	1278973	16101758	hadcm3n_82z5_1980_40_008461796_1	25,920	75,646	2.9184