Task 15227723

Name	hadam3p_eu_wblw_1979_1_006819996_1
Workunit	7023312
Created	3 Sep 2012, 8:29:38 UTC
Sent	18 Sep 2012, 11:55:23 UTC
Report deadline	31 Aug 2013, 17:15:23 UTC
Received	26 Sep 2012, 6:26:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1235277
Run time	3 days 15 hours 14 min 45 sec
CPU time	3 days 4 hours 55 min 44 sec
Validate state	Invalid
Credit	1,591.48
Device peak FLOPS	2.16 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6296, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8664, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 08:48:43 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:56 (5416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:34:04 (6452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:34:05 (6452): No heartbeat from core client for 30 sec - exiting 10:34:06 (6452): No heartbeat from core client for 30 sec - exiting 10:34:07 (6452): No heartbeat from core client for 30 sec - exiting 10:34:08 (6452): No heartbeat from core client for 30 sec - exiting 10:34:09 (6452): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6524, selfPID=6524, iMonCtr=2 11:46:21 (7116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:58:26 (7236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:58:27 (7236): No heartbeat from core client for 30 sec - exiting 11:58:28 (7236): No heartbeat from core client for 30 sec - exiting 11:58:29 (7236): No heartbeat from core client for 30 sec - exiting 11:58:30 (7236): No heartbeat from core client for 30 sec - exiting 12:34:40 (7072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:53:17 (7428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:38:39 (7212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:05:54 (5800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:09 (7936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1680, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... 06:29:11 (6276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:06:27 (4940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:28 (4940): No heartbeat from core client for 30 sec - exiting 20:09:51 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:10 (9624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:11 (9624): No heartbeat from core client for 30 sec - exiting 21:13:12 (9624): No heartbeat from core client for 30 sec - exiting 21:13:13 (9624): No heartbeat from core client for 30 sec - exiting 21:13:14 (9624): No heartbeat from core client for 30 sec - exiting 21:13:15 (9624): No heartbeat from core client for 30 sec - exiting 21:13:16 (9624): No heartbeat from core client for 30 sec - exiting 23:01:38 (5612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:17:10 (8660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:17:11 (8660): No heartbeat from core client for 30 sec - exiting 01:17:12 (8660): No heartbeat from core client for 30 sec - exiting 01:17:13 (8660): No heartbeat from core client for 30 sec - exiting 01:17:14 (8660): No heartbeat from core client for 30 sec - exiting 01:17:15 (8660): No heartbeat from core client for 30 sec - exiting 01:17:16 (8660): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3580, iMonCtr=2 Model crash detected, will try to restart... 08:09:23 (6576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:09:24 (6576): No heartbeat from core client for 30 sec - exiting 08:09:25 (6576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5104, selfPID=5104, iMonCtr=2 15:32:29 (4380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:32:30 (4380): No heartbeat from core client for 30 sec - exiting 15:32:31 (4380): No heartbeat from core client for 30 sec - exiting 15:32:32 (4380): No heartbeat from core client for 30 sec - exiting 15:32:33 (4380): No heartbeat from core client for 30 sec - exiting 15:32:34 (4380): No heartbeat from core client for 30 sec - exiting 15:32:35 (4380): No heartbeat from core client for 30 sec - exiting 15:59:42 (812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:59:43 (812): No heartbeat from core client for 30 sec - exiting 16:02:44 (1104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:48 (5200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:49 (5200): No heartbeat from core client for 30 sec - exiting 16:23:50 (5200): No heartbeat from core client for 30 sec - exiting 16:23:51 (5200): No heartbeat from core client for 30 sec - exiting 16:23:52 (5200): No heartbeat from core client for 30 sec - exiting 16:23:53 (5200): No heartbeat from core client for 30 sec - exiting 16:23:54 (5200): No heartbeat from core client for 30 sec - exiting 16:23:55 (5200): No heartbeat from core client for 30 sec - exiting 18:59:06 (5672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:05:09 (5648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:18 (3808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:19 (3808): No heartbeat from core client for 30 sec - exiting 19:20:20 (3808): No heartbeat from core client for 30 sec - exiting 19:20:21 (3808): No heartbeat from core client for 30 sec - exiting 19:20:22 (3808): No heartbeat from core client for 30 sec - exiting 19:20:23 (3808): No heartbeat from core client for 30 sec - exiting 19:20:24 (3808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:36:30 (5408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:09:45 (7632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:09:46 (7632): No heartbeat from core client for 30 sec - exiting 12:09:47 (7632): No heartbeat from core client for 30 sec - exiting 12:09:48 (7632): No heartbeat from core client for 30 sec - exiting 12:09:49 (7632): No heartbeat from core client for 30 sec - exiting 12:09:50 (7632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4960, selfPID=4960, iMonCtr=2 16:33:44 (2364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:42:04 (3160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:13 (1788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:10:37 (7124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5636, selfPID=5636, iMonCtr=2 13:13:55 (3624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:19:59 (2524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5248, selfPID=5248, iMonCtr=2 13:51:25 (6404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:21:45 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:52 (2152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:53 (2152): No heartbeat from core client for 30 sec - exiting 16:21:54 (2152): No heartbeat from core client for 30 sec - exiting 16:21:55 (2152): No heartbeat from core client for 30 sec - exiting 16:21:56 (2152): No heartbeat from core client for 30 sec - exiting 16:21:57 (2152): No heartbeat from core client for 30 sec - exiting 17:10:24 (7240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:58 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5248, selfPID=5248, iMonCtr=2 17:38:09 (872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7976, selfPID=7976, iMonCtr=2 18:58:01 (3012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:11 (6420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:12 (6420): No heartbeat from core client for 30 sec - exiting 19:13:13 (6420): No heartbeat from core client for 30 sec - exiting 19:13:14 (6420): No heartbeat from core client for 30 sec - exiting 19:13:15 (6420): No heartbeat from core client for 30 sec - exiting 19:16:19 (3616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:32 (3096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:33 (3096): No heartbeat from core client for 30 sec - exiting 19:34:34 (3096): No heartbeat from core client for 30 sec - exiting 19:34:35 (3096): No heartbeat from core client for 30 sec - exiting 19:34:36 (3096): No heartbeat from core client for 30 sec - exiting 19:34:37 (3096): No heartbeat from core client for 30 sec - exiting 19:37:41 (7424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:57:14 (3624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:21:24 (7328): No heartbeat from core client for 30 sec - exiting 21:21:25 (7328): No heartbeat from core client for 30 sec - exiting 21:21:26 (7328): No heartbeat from core client for 30 sec - exiting 21:21:27 (7328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:27 (3896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... RSuspended CPDN Monitor - Suspend request from BOINC... 01:26:54 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:29:57 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:29:58 (4596): No heartbeat from core client for 30 sec - exiting 01:29:59 (4596): No heartbeat from core client for 30 sec - exiting 01:30:00 (4596): No heartbeat from core client for 30 sec - exiting 01:30:01 (4596): No heartbeat from core client for 30 sec - exiting 02:20:24 (2872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:54:05 (8816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:30:39 (8876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:30:40 (8876): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Sep 2012 03:26:26	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	92,256	272,620	2.9550
25 Sep 2012 13:23:59	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	80,736	237,750	2.9448
23 Sep 2012 19:34:54	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	69,216	203,141	2.9349
22 Sep 2012 23:38:54	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	57,696	167,980	2.9115
21 Sep 2012 23:45:55	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	46,176	133,611	2.8935
20 Sep 2012 04:30:38	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	34,656	99,876	2.8819
19 Sep 2012 18:44:45	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	23,136	66,835	2.8888
19 Sep 2012 09:08:13	1235277	15227723	hadam3p_eu_wblw_1979_1_006819996_1	11,616	33,422	2.8772