Task 14787184

Name	hadam3p_eu_d5cv_2002_1_007973153_1
Workunit	8128267
Created	11 Jun 2012, 22:47:20 UTC
Sent	11 Jun 2012, 22:47:27 UTC
Report deadline	25 May 2013, 4:07:27 UTC
Received	22 Jun 2012, 3:00:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1221385
Run time	7 days 4 hours 58 min 11 sec
CPU time	6 days 8 hours 19 min 38 sec
Validate state	Invalid
Credit	2,187.67
Device peak FLOPS	1.40 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:52:29 (7060): No heartbeat from core client for 30 sec - exiting 09:52:30 (7060): No heartbeat from core client for 30 sec - exiting 09:52:31 (7060): No heartbeat from core client for 30 sec - exiting 09:52:32 (7060): No heartbeat from core client for 30 sec - exiting 09:52:33 (7060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1776, selfPID=7480, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6628, selfPID=1084, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:17:51 (4264): No heartbeat from core client for 30 sec - exiting 07:17:52 (4264): No heartbeat from core client for 30 sec - exiting 07:17:53 (4264): No heartbeat from core client for 30 sec - exiting 07:17:54 (4264): No heartbeat from core client for 30 sec - exiting 07:17:55 (4264): No heartbeat from core client for 30 sec - exiting 07:17:56 (4264): No heartbeat from core client for 30 sec - exiting 07:17:57 (4264): No heartbeat from core client for 30 sec - exiting 07:17:58 (4264): No heartbeat from core client for 30 sec - exiting 07:17:59 (4264): No heartbeat from core client for 30 sec - exiting 07:18:00 (4264): No heartbeat from core client for 30 sec - exiting 07:18:02 (4264): No heartbeat from core client for 30 sec - exiting 07:18:03 (4264): No heartbeat from core client for 30 sec - exiting 07:18:04 (4264): No heartbeat from core client for 30 sec - exiting 07:18:05 (4264): No heartbeat from core client for 30 sec - exiting 07:18:06 (4264): No heartbeat from core client for 30 sec - exiting 07:18:07 (4264): No heartbeat from core client for 30 sec - exiting 07:18:08 (4264): No heartbeat from core client for 30 sec - exiting 07:18:09 (4264): No heartbeat from core client for 30 sec - exiting 07:18:10 (4264): No heartbeat from core client for 30 sec - exiting 07:18:11 (4264): No heartbeat from core client for 30 sec - exiting 07:18:12 (4264): No heartbeat from core client for 30 sec - exiting 07:18:14 (4264): No heartbeat from core client for 30 sec - exiting 07:18:15 (4264): No heartbeat from core client for 30 sec - exiting 07:18:16 (4264): No heartbeat from core client for 30 sec - exiting 07:18:17 (4264): No heartbeat from core client for 30 sec - exiting 07:18:18 (4264): No heartbeat from core client for 30 sec - exiting 07:18:19 (4264): No heartbeat from core client for 30 sec - exiting 07:18:20 (4264): No heartbeat from core client for 30 sec - exiting 07:18:21 (4264): No heartbeat from core client for 30 sec - exiting 07:18:22 (4264): No heartbeat from core client for 30 sec - exiting 07:18:23 (4264): No heartbeat from core client for 30 sec - exiting 07:18:25 (4264): No heartbeat from core client for 30 sec - exiting 07:18:26 (4264): No heartbeat from core client for 30 sec - exiting 07:18:27 (4264): No heartbeat from core client for 30 sec - exiting 07:18:28 (4264): No heartbeat from core client for 30 sec - exiting 07:18:29 (4264): No heartbeat from core client for 30 sec - exiting 07:18:30 (4264): No heartbeat from core client for 30 sec - exiting 07:18:31 (4264): No heartbeat from core client for 30 sec - exiting 07:18:32 (4264): No heartbeat from core client for 30 sec - exiting 07:18:33 (4264): No heartbeat from core client for 30 sec - exiting 07:18:34 (4264): No heartbeat from core client for 30 sec - exiting 07:18:35 (4264): No heartbeat from core client for 30 sec - exiting 07:18:37 (4264): No heartbeat from core client for 30 sec - exiting 07:18:38 (4264): No heartbeat from core client for 30 sec - exiting 07:18:39 (4264): No heartbeat from core client for 30 sec - exiting 07:18:40 (4264): No heartbeat from core client for 30 sec - exiting 07:18:41 (4264): No heartbeat from core client for 30 sec - exiting 07:18:42 (4264): No heartbeat from core client for 30 sec - exiting 07:18:43 (4264): No heartbeat from core client for 30 sec - exiting 07:18:44 (4264): No heartbeat from core client for 30 sec - exiting 07:18:45 (4264): No heartbeat from core client for 30 sec - exiting 07:18:46 (4264): No heartbeat from core client for 30 sec - exiting 07:18:47 (4264): No heartbeat from core client for 30 sec - exiting 07:18:49 (4264): No heartbeat from core client for 30 sec - exiting 07:18:50 (4264): No heartbeat from core client for 30 sec - exiting 07:18:51 (4264): No heartbeat from core client for 30 sec - exiting 07:18:52 (4264): No heartbeat from core client for 30 sec - exiting 07:18:53 (4264): No heartbeat from core client for 30 sec - exiting 07:18:54 (4264): No heartbeat from core client for 30 sec - exiting 07:18:55 (4264): No heartbeat from core client for 30 sec - exiting 07:18:56 (4264): No heartbeat from core client for 30 sec - exiting 07:18:57 (4264): No heartbeat from core client for 30 sec - exiting 07:18:58 (4264): No heartbeat from core client for 30 sec - exiting 07:18:59 (4264): No heartbeat from core client for 30 sec - exiting 07:19:01 (4264): No heartbeat from core client for 30 sec - exiting 07:19:02 (4264): No heartbeat from core client for 30 sec - exiting 07:19:03 (4264): No heartbeat from core client for 30 sec - exiting 07:19:04 (4264): No heartbeat from core client for 30 sec - exiting 07:19:05 (4264): No heartbeat from core client for 30 sec - exiting 07:19:06 (4264): No heartbeat from core client for 30 sec - exiting 07:19:07 (4264): No heartbeat from core client for 30 sec - exiting 07:19:08 (4264): No heartbeat from core client for 30 sec - exiting 07:19:09 (4264): No heartbeat from core client for 30 sec - exiting 07:19:10 (4264): No heartbeat from core client for 30 sec - exiting 07:19:11 (4264): No heartbeat from core client for 30 sec - exiting 07:19:13 (4264): No heartbeat from core client for 30 sec - exiting 07:19:14 (4264): No heartbeat from core client for 30 sec - exiting 07:19:15 (4264): No heartbeat from core client for 30 sec - exiting 07:19:16 (4264): No heartbeat from core client for 30 sec - exiting 07:19:17 (4264): No heartbeat from core client for 30 sec - exiting 07:19:18 (4264): No heartbeat from core client for 30 sec - exiting 07:19:19 (4264): No heartbeat from core client for 30 sec - exiting 07:19:20 (4264): No heartbeat from core client for 30 sec - exiting 07:19:21 (4264): No heartbeat from core client for 30 sec - exiting 07:19:22 (4264): No heartbeat from core client for 30 sec - exiting 07:19:23 (4264): No heartbeat from core client for 30 sec - exiting 07:19:25 (4264): No heartbeat from core client for 30 sec - exiting 07:19:26 (4264): No heartbeat from core client for 30 sec - exiting 07:19:27 (4264): No heartbeat from core client for 30 sec - exiting 07:19:28 (4264): No heartbeat from core client for 30 sec - exiting 07:19:29 (4264): No heartbeat from core client for 30 sec - exiting 07:19:30 (4264): No heartbeat from core client for 30 sec - exiting 07:19:31 (4264): No heartbeat from core client for 30 sec - exiting 07:19:32 (4264): No heartbeat from core client for 30 sec - exiting 07:19:33 (4264): No heartbeat from core client for 30 sec - exiting 07:19:34 (4264): No heartbeat from core client for 30 sec - exiting 07:19:35 (4264): No heartbeat from core client for 30 sec - exiting 07:19:37 (4264): No heartbeat from core client for 30 sec - exiting 07:19:38 (4264): No heartbeat from core client for 30 sec - exiting 07:19:39 (4264): No heartbeat from core client for 30 sec - exiting 07:19:40 (4264): No heartbeat from core client for 30 sec - exiting 07:19:41 (4264): No heartbeat from core client for 30 sec - exiting 07:19:42 (4264): No heartbeat from core client for 30 sec - exiting 07:19:43 (4264): No heartbeat from core client for 30 sec - exiting 07:19:44 (4264): No heartbeat from core client for 30 sec - exiting 07:19:45 (4264): No heartbeat from core client for 30 sec - exiting 07:19:46 (4264): No heartbeat from core client for 30 sec - exiting 07:19:47 (4264): No heartbeat from core client for 30 sec - exiting 07:19:49 (4264): No heartbeat from core client for 30 sec - exiting 07:19:50 (4264): No heartbeat from core client for 30 sec - exiting 07:19:51 (4264): No heartbeat from core client for 30 sec - exiting 07:19:52 (4264): No heartbeat from core client for 30 sec - exiting 07:19:53 (4264): No heartbeat from core client for 30 sec - exiting 07:19:54 (4264): No heartbeat from core client for 30 sec - exiting 07:19:55 (4264): No heartbeat from core client for 30 sec - exiting 07:19:56 (4264): No heartbeat from core client for 30 sec - exiting 07:19:57 (4264): No heartbeat from core client for 30 sec - exiting 07:19:58 (4264): No heartbeat from core client for 30 sec - exiting 07:20:00 (4264): No heartbeat from core client for 30 sec - exiting 07:20:01 (4264): No heartbeat from core client for 30 sec - exiting 07:20:02 (4264): No heartbeat from core client for 30 sec - exiting 07:20:03 (4264): No heartbeat from core client for 30 sec - exiting 07:20:04 (4264): No heartbeat from core client for 30 sec - exiting 07:20:05 (4264): No heartbeat from core client for 30 sec - exiting 07:20:06 (4264): No heartbeat from core client for 30 sec - exiting 07:20:07 (4264): No heartbeat from core client for 30 sec - exiting 07:20:08 (4264): No heartbeat from core client for 30 sec - exiting 07:20:09 (4264): No heartbeat from core client for 30 sec - exiting 07:20:10 (4264): No heartbeat from core client for 30 sec - exiting 07:20:12 (4264): No heartbeat from core client for 30 sec - exiting 07:20:13 (4264): No heartbeat from core client for 30 sec - exiting 07:20:14 (4264): No heartbeat from core client for 30 sec - exiting 07:20:15 (4264): No heartbeat from core client for 30 sec - exiting 07:20:16 (4264): No heartbeat from core client for 30 sec - exiting 07:20:17 (4264): No heartbeat from core client for 30 sec - exiting 07:20:18 (4264): No heartbeat from core client for 30 sec - exiting 07:20:19 (4264): No heartbeat from core client for 30 sec - exiting 07:20:20 (4264): No heartbeat from core client for 30 sec - exiting 07:20:21 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:20:22 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6472, selfPID=6472, iMonCtr=2 14:37:57 (6740): No heartbeat from core client for 30 sec - exiting 14:37:58 (6740): No heartbeat from core client for 30 sec - exiting 14:37:59 (6740): No heartbeat from core client for 30 sec - exiting 14:38:00 (6740): No heartbeat from core client for 30 sec - exiting 14:38:01 (6740): No heartbeat from core client for 30 sec - exiting 14:38:02 (6740): No heartbeat from core client for 30 sec - exiting 14:38:03 (6740): No heartbeat from core client for 30 sec - exiting 14:38:04 (6740): No heartbeat from core client for 30 sec - exiting 14:38:06 (6740): No heartbeat from core client for 30 sec - exiting 14:38:07 (6740): No heartbeat from core client for 30 sec - exiting 14:38:08 (6740): No heartbeat from core client for 30 sec - exiting 14:38:09 (6740): No heartbeat from core client for 30 sec - exiting 14:38:10 (6740): No heartbeat from core client for 30 sec - exiting 14:38:11 (6740): No heartbeat from core client for 30 sec - exiting 14:38:12 (6740): No heartbeat from core client for 30 sec - exiting 14:38:13 (6740): No heartbeat from core client for 30 sec - exiting 14:38:14 (6740): No heartbeat from core client for 30 sec - exiting 14:38:15 (6740): No heartbeat from core client for 30 sec - exiting 14:38:16 (6740): No heartbeat from core client for 30 sec - exiting 14:38:18 (6740): No heartbeat from core client for 30 sec - exiting 14:38:19 (6740): No heartbeat from core client for 30 sec - exiting 14:38:20 (6740): No heartbeat from core client for 30 sec - exiting 14:38:21 (6740): No heartbeat from core client for 30 sec - exiting 14:38:22 (6740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=380, selfPID=380, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4792, selfPID=4792, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:46:57 (4376): No heartbeat from core client for 30 sec - exiting 09:46:59 (4376): No heartbeat from core client for 30 sec - exiting 09:47:00 (4376): No heartbeat from core client for 30 sec - exiting 09:47:01 (4376): No heartbeat from core client for 30 sec - exiting 09:47:02 (4376): No heartbeat from core client for 30 sec - exiting 09:47:03 (4376): No heartbeat from core client for 30 sec - exiting 09:47:04 (4376): No heartbeat from core client for 30 sec - exiting 09:47:05 (4376): No heartbeat from core client for 30 sec - exiting 09:47:06 (4376): No heartbeat from core client for 30 sec - exiting 09:47:07 (4376): No heartbeat from core client for 30 sec - exiting 09:47:09 (4376): No heartbeat from core client for 30 sec - exiting 09:47:10 (4376): No heartbeat from core client for 30 sec - exiting 09:47:11 (4376): No heartbeat from core client for 30 sec - exiting 09:47:12 (4376): No heartbeat from core client for 30 sec - exiting 09:47:13 (4376): No heartbeat from core client for 30 sec - exiting 09:47:14 (4376): No heartbeat from core client for 30 sec - exiting 09:47:15 (4376): No heartbeat from core client for 30 sec - exiting 09:47:16 (4376): No heartbeat from core client for 30 sec - exiting 09:47:17 (4376): No heartbeat from core client for 30 sec - exiting 09:47:18 (4376): No heartbeat from core client for 30 sec - exiting 09:47:19 (4376): No heartbeat from core client for 30 sec - exiting 09:47:21 (4376): No heartbeat from core client for 30 sec - exiting 09:47:22 (4376): No heartbeat from core client for 30 sec - exiting 09:47:23 (4376): No heartbeat from core client for 30 sec - exiting 09:47:24 (4376): No heartbeat from core client for 30 sec - exiting 09:47:25 (4376): No heartbeat from core client for 30 sec - exiting 09:47:26 (4376): No heartbeat from core client for 30 sec - exiting 09:47:27 (4376): No heartbeat from core client for 30 sec - exiting 09:47:28 (4376): No heartbeat from core client for 30 sec - exiting 09:47:29 (4376): No heartbeat from core client for 30 sec - exiting 09:47:30 (4376): No heartbeat from core client for 30 sec - exiting 09:47:31 (4376): No heartbeat from core client for 30 sec - exiting 09:47:33 (4376): No heartbeat from core client for 30 sec - exiting 09:47:34 (4376): No heartbeat from core client for 30 sec - exiting 09:47:35 (4376): No heartbeat from core client for 30 sec - exiting 09:47:36 (4376): No heartbeat from core client for 30 sec - exiting 09:47:37 (4376): No heartbeat from core client for 30 sec - exiting 09:47:38 (4376): No heartbeat from core client for 30 sec - exiting 09:47:39 (4376): No heartbeat from core client for 30 sec - exiting 09:47:40 (4376): No heartbeat from core client for 30 sec - exiting 09:47:41 (4376): No heartbeat from core client for 30 sec - exiting 09:47:42 (4376): No heartbeat from core client for 30 sec - exiting 09:47:43 (4376): No heartbeat from core client for 30 sec - exiting 09:47:45 (4376): No heartbeat from core client for 30 sec - exiting 09:47:46 (4376): No heartbeat from core client for 30 sec - exiting 09:47:47 (4376): No heartbeat from core client for 30 sec - exiting 09:47:48 (4376): No heartbeat from core client for 30 sec - exiting 09:47:49 (4376): No heartbeat from core client for 30 sec - exiting 09:47:50 (4376): No heartbeat from core client for 30 sec - exiting 09:47:51 (4376): No heartbeat from core client for 30 sec - exiting 09:47:52 (4376): No heartbeat from core client for 30 sec - exiting 09:47:53 (4376): No heartbeat from core client for 30 sec - exiting 09:47:54 (4376): No heartbeat from core client for 30 sec - exiting 09:47:55 (4376): No heartbeat from core client for 30 sec - exiting 09:47:57 (4376): No heartbeat from core client for 30 sec - exiting 09:47:58 (4376): No heartbeat from core client for 30 sec - exiting 09:47:59 (4376): No heartbeat from core client for 30 sec - exiting 09:48:00 (4376): No heartbeat from core client for 30 sec - exiting 09:48:01 (4376): No heartbeat from core client for 30 sec - exiting 09:48:02 (4376): No heartbeat from core client for 30 sec - exiting 09:48:03 (4376): No heartbeat from core client for 30 sec - exiting 09:48:04 (4376): No heartbeat from core client for 30 sec - exiting 09:48:05 (4376): No heartbeat from core client for 30 sec - exiting 09:48:06 (4376): No heartbeat from core client for 30 sec - exiting 09:48:07 (4376): No heartbeat from core client for 30 sec - exiting 09:48:09 (4376): No heartbeat from core client for 30 sec - exiting 09:48:10 (4376): No heartbeat from core client for 30 sec - exiting 09:48:11 (4376): No heartbeat from core client for 30 sec - exiting 09:48:12 (4376): No heartbeat from core client for 30 sec - exiting 09:48:13 (4376): No heartbeat from core client for 30 sec - exiting 09:48:14 (4376): No heartbeat from core client for 30 sec - exiting 09:48:15 (4376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3512, selfPCPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Jun 2012 13:11:26	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	126,816	523,454	4.1277
20 Jun 2012 13:59:30	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	115,296	475,083	4.1206
19 Jun 2012 15:53:53	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	103,776	428,619	4.1302
18 Jun 2012 17:04:10	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	92,256	381,512	4.1354
17 Jun 2012 01:18:23	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	80,737	335,684	4.1577
16 Jun 2012 15:47:05	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	80,736	335,079	4.1503
16 Jun 2012 01:42:39	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	69,216	288,526	4.1685
15 Jun 2012 11:27:46	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	57,696	242,047	4.1952
14 Jun 2012 21:53:20	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	46,176	194,786	4.2183
14 Jun 2012 04:32:45	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	34,656	147,259	4.2492
13 Jun 2012 13:50:21	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	23,136	99,099	4.2833
12 Jun 2012 16:04:02	1221385	14787184	hadam3p_eu_d5cv_2002_1_007973153_1	11,616	50,497	4.3472