Task 13979321

Name	hadam3p_pnw_8sm0_2003_1_007710036_0
Workunit	7865144
Created	25 Jan 2012, 19:04:17 UTC
Sent	3 Feb 2012, 10:39:20 UTC
Report deadline	15 Jan 2013, 15:59:20 UTC
Received	26 Mar 2012, 9:44:34 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	945967
Run time	8 days 8 hours 40 min 33 sec
CPU time	8 days 8 hours 40 min 33 sec
Validate state	Workunit error - check skipped
Credit	3,005.88
Device peak FLOPS	1.99 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <stderr_txt> Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3076, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4604, iMonCtr=2 Model crash detected, will try to restart... 12:29:18 (5320): No heartbeat from core client for 30 sec - exiting 12:29:19 (5320): No heartbeat from core client for 30 sec - exiting 12:29:20 (5320): No heartbeat from core client for 30 sec - exiting 12:29:21 (5320): No heartbeat from core client for 30 sec - exiting 12:29:22 (5320): No heartbeat from core client for 30 sec - exiting 12:29:23 (5320): No heartbeat from core client for 30 sec - exiting 12:29:24 (5320): No heartbeat from core client for 30 sec - exiting 12:29:25 (5320): No heartbeat from core client for 30 sec - exiting 12:29:26 (5320): No heartbeat from core client for 30 sec - exiting 12:29:27 (5320): No heartbeat from core client for 30 sec - exiting 12:29:28 (5320): No heartbeat from core client for 30 sec - exiting 12:29:29 (5320): No heartbeat from core client for 30 sec - exiting 12:29:30 (5320): No heartbeat from core client for 30 sec - exiting 12:29:31 (5320): No heartbeat from core client for 30 sec - exiting 12:29:32 (5320): No heartbeat from core client for 30 sec - exiting 12:29:33 (5320): No heartbeat from core client for 30 sec - exiting 12:29:34 (5320): No heartbeat from core client for 30 sec - exiting 12:29:35 (5320): No heartbeat from core client for 30 sec - exiting 12:29:36 (5320): No heartbeat from core client for 30 sec - exiting 12:29:37 (5320): No heartbeat from core client for 30 sec - exiting 12:29:38 (5320): No heartbeat from core client for 30 sec - exiting 12:29:39 (5320): No heartbeat from core client for 30 sec - exiting 12:29:40 (5320): No heartbeat from core client for 30 sec - exiting 12:29:41 (5320): No heartbeat from core client for 30 sec - exiting 12:29:42 (5320): No heartbeat from core client for 30 sec - exiting 12:29:43 (5320): No heartbeat from core client for 30 sec - exiting 12:29:44 (5320): No heartbeat from core client for 30 sec - exiting 12:29:45 (5320): No heartbeat from core client for 30 sec - exiting 12:29:46 (5320): No heartbeat from core client for 30 sec - exiting 12:29:47 (5320): No heartbeat from core client for 30 sec - exiting 12:29:48 (5320): No heartbeat from core client for 30 sec - exiting 12:29:49 (5320): No heartbeat from core client for 30 sec - exiting 12:29:50 (5320): No heartbeat from core client for 30 sec - exiting 12:29:51 (5320): No heartbeat from core client for 30 sec - exiting 12:29:52 (5320): No heartbeat from core client for 30 sec - exiting 12:29:53 (5320): No heartbeat from core client for 30 sec - exiting 12:29:54 (5320): No heartbeat from core client for 30 sec - exiting 12:29:55 (5320): No heartbeat from core client for 30 sec - exiting 12:29:56 (5320): No heartbeat from core client for 30 sec - exiting 12:29:57 (5320): No heartbeat from core client for 30 sec - exiting 12:29:58 (5320): No heartbeat from core client for 30 sec - exiting 12:29:59 (5320): No heartbeat from core client for 30 sec - exiting 12:30:00 (5320): No heartbeat from core client for 30 sec - exiting 12:30:01 (5320): No heartbeat from core client for 30 sec - exiting 12:30:02 (5320): No heartbeat from core client for 30 sec - exiting 12:30:03 (5320): No heartbeat from core client for 30 sec - exiting 12:30:04 (5320): No heartbeat from core client for 30 sec - exiting 12:30:05 (5320): No heartbeat from core client for 30 sec - exiting 12:30:06 (5320): No heartbeat from core client for 30 sec - exiting 12:30:07 (5320): No heartbeat from core client for 30 sec - exiting 12:30:08 (5320): No heartbeat from core client for 30 sec - exiting 12:30:09 (5320): No heartbeat from core client for 30 sec - exiting 12:30:10 (5320): No heartbeat from core client for 30 sec - exiting 12:30:11 (5320): No heartbeat from core client for 30 sec - exiting 12:30:12 (5320): No heartbeat from core client for 30 sec - exiting 12:30:13 (5320): No heartbeat from core client for 30 sec - exiting 12:30:14 (5320): No heartbeat from core client for 30 sec - exiting 12:30:15 (5320): No heartbeat from core client for 30 sec - exiting 12:30:16 (5320): No heartbeat from core client for 30 sec - exiting 12:30:17 (5320): No heartbeat from core client for 30 sec - exiting 12:30:18 (5320): No heartbeat from core client for 30 sec - exiting 12:30:19 (5320): No heartbeat from core client for 30 sec - exiting 12:30:20 (5320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6524, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7144, selfPID=5420, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6388, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5916, selfPID=4868, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5424, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7756, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7772, selfPID=1532, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 3 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7536, selfPID=1480, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 3 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5904, selfPID=5716, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6864, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6516, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 5 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6396, selfPID=2268, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 5 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6756, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 5 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5828, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1352, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 11:03:04 (6448): No heartbeat from core client for 30 sec - exiting 11:03:06 (6448): No heartbeat from core client for 30 sec - exiting 11:03:08 (6448): No heartbeat from core client for 30 sec - exiting 11:03:09 (6448): No heartbeat from core client for 30 sec - exiting 11:03:10 (6448): No heartbeat from core client for 30 sec - exiting 11:03:11 (6448): No heartbeat from core client for 30 sec - exiting 11:03:12 (6448): No heartbeat from core client for 30 sec - exiting 11:03:13 (6448): No heartbeat from core client for 30 sec - exiting 11:03:14 (6448): No heartbeat from core client for 30 sec - exiting 11:03:15 (6448): No heartbeat from core client for 30 sec - exiting 11:03:16 (6448): No heartbeat from core client for 30 sec - exiting 11:03:17 (6448): No heartbeat from core client for 30 sec - exiting 11:03:19 (6448): No heartbeat from core client for 30 sec - exiting 11:03:20 (6448): No heartbeat from core client for 30 sec - exiting 11:03:21 (6448): No heartbeat from core client for 30 sec - exiting 11:03:22 (6448): No heartbeat from core client for 30 sec - exiting 11:03:23 (6448): No heartbeat from core client for 30 sec - exiting 11:03:24 (6448): No heartbeat from core client for 30 sec - exiting 11:03:25 (6448): No heartbeat from core client for 30 sec - exiting 11:03:26 (6448): No heartbeat from core client for 30 sec - exiting 11:03:27 (6448): No heartbeat from core client for 30 sec - exiting 11:03:28 (6448): No heartbeat from core client for 30 sec - exiting 11:03:29 (6448): No heartbeat from core client for 30 sec - exiting 11:03:33 (6448): No heartbeat from core client for 30 sec - exiting 11:03:35 (6448): No heartbeat from core client for 30 sec - exiting 11:03:38 (6448): No heartbeat from core client for 30 sec - exiting 11:03:39 (6448): No heartbeat from core client for 30 sec - exiting 11:03:40 (6448): No heartbeat from core client for 30 sec - exiting 11:03:42 (6448): No heartbeat from core client for 30 sec - exiting 11:03:44 (6448): No heartbeat from core client for 30 sec - exiting 11:03:45 (6448): No heartbeat from core client for 30 sec - exiting 11:03:46 (6448): No heartbeat from core client for 30 sec - exiting 11:03:47 (6448): No heartbeat from core client for 30 sec - exiting 11:03:48 (6448): No heartbeat from core client for 30 sec - exiting 11:03:49 (6448): No heartbeat from core client for 30 sec - exiting 11:03:50 (6448): No heartbeat from core client for 30 sec - exiting 11:03:51 (6448): No heartbeat from core client for 30 sec - exiting 11:03:53 (6448): No heartbeat from core client for 30 sec - exiting 11:03:54 (6448): No heartbeat from core client for 30 sec - exiting 11:03:55 (6448): No heartbeat from core client for 30 sec - exiting 11:03:56 (6448): No heartbeat from core client for 30 sec - exiting 11:03:57 (6448): No heartbeat from core client for 30 sec - exiting 11:03:58 (6448): No heartbeat from core client for 30 sec - exiting 11:03:59 (6448): No heartbeat from core client for 30 sec - exiting 11:04:00 (6448): No heartbeat from core client for 30 sec - exiting 11:04:01 (6448): No heartbeat from core client for 30 sec - exiting 11:04:02 (6448): No heartbeat from core client for 30 sec - exiting 11:04:03 (6448): No heartbeat from core client for 30 sec - exiting 11:04:05 (6448): No heartbeat from core client for 30 sec - exiting 11:04:06 (6448): No heartbeat from core client for 30 sec - exiting 11:04:07 (6448): No heartbeat from core client for 30 sec - exiting 11:04:08 (6448): No heartbeat from core client for 30 sec - exiting 11:04:09 (6448): No heartbeat from core client for 30 sec - exiting 11:04:10 (6448): No heartbeat from core client for 30 sec - exiting 11:04:11 (6448): No heartbeat from core client for 30 sec - exiting 11:04:12 (6448): No heartbeat from core client for 30 sec - exiting 11:04:13 (6448): No heartbeat from core client for 30 sec - exiting 11:04:14 (6448): No heartbeat from core client for 30 sec - exiting 11:04:15 (6448): No heartbeat from core client for 30 sec - exiting 11:04:16 (6448): No heartbeat from core client for 30 sec - exiting 11:04:17 (6448): No heartbeat from core client for 30 sec - exiting 11:04:18 (6448): No heartbeat from core client for 30 sec - exiting 11:04:19 (6448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:04:20 (6448): No heartbeat from core client for 30 sec - exiting 12:04:59 (7416): No heartbeat from core client for 30 sec - exiting 12:05:00 (7416): No heartbeat from core client for 30 sec - exiting 12:05:01 (7416): No heartbeat from core client for 30 sec - exiting 12:05:02 (7416): No heartbeat from core client for 30 sec - exiting 12:05:03 (7416): No heartbeat from core client for 30 sec - exiting 12:05:04 (7416): No heartbeat from core client for 30 sec - exiting 12:05:06 (7416): No heartbeat from core client for 30 sec - exiting 12:05:07 (7416): No heartbeat from core client for 30 sec - exiting 12:05:08 (7416): No heartbeat from core client for 30 sec - exiting 12:05:09 (7416): No heartbeat from core client for 30 sec - exiting 12:05:10 (7416): No heartbeat from core client for 30 sec - exiting 12:05:12 (7416): No heartbeat from core client for 30 sec - exiting 12:05:13 (7416): No heartbeat from core client for 30 sec - exiting 12:05:14 (7416): No heartbeat from core client for 30 sec - exiting 12:05:15 (7416): No heartbeat from core client for 30 sec - exiting 12:05:16 (7416): No heartbeat from core client for 30 sec - exiting 12:05:17 (7416): No heartbeat from core client for 30 sec - exiting 12:05:18 (7416): No heartbeat from core client for 30 sec - exiting 12:05:19 (7416): No heartbeat from core client for 30 sec - exiting 12:05:20 (7416): No heartbeat from core client for 30 sec - exiting 12:05:21 (7416): No heartbeat from core client for 30 sec - exiting 12:05:22 (7416): No heartbeat from core client for 30 sec - exiting 12:05:23 (7416): No heartbeat from core client for 30 sec - exiting 12:05:24 (7416): No heartbeat from core client for 30 sec - exiting 12:05:25 (7416): No heartbeat from core client for 30 sec - exiting 12:05:26 (7416): No heartbeat from core client for 30 sec - exiting 12:05:27 (7416): No heartbeat from core client for 30 sec - exiting 12:05:28 (7416): No heartbeat from core client for 30 sec - exiting 12:05:29 (7416): No heartbeat from core client for 30 sec - exiting 12:05:30 (7416): No heartbeat from core client for 30 sec - exiting 12:05:31 (7416): No heartbeat from core client for 30 sec - exiting 12:05:32 (7416): No heartbeat from core client for 30 sec - exiting 12:05:33 (7416): No heartbeat from core client for 30 sec - exiting 12:05:34 (7416): No heartbeat from core client for 30 sec - exiting 12:05:35 (7416): No heartbeat from core client for 30 sec - exiting 12:05:36 (7416): No heartbeat from core client for 30 sec - exiting 12:05:37 (7416): No heartbeat from core client for 30 sec - exiting 12:05:38 (7416): No heartbeat from core client for 30 sec - exiting 12:05:39 (7416): No heartbeat from core client for 30 sec - exiting 12:05:40 (7416): No heartbeat from core client for 30 sec - exiting 12:05:41 (7416): No heartbeat from core client for 30 sec - exiting 12:05:42 (7416): No heartbeat from core client for 30 sec - exiting 12:05:43 (7416): No heartbeat from core client for 30 sec - exiting 12:05:44 (7416): No heartbeat from core client for 30 sec - exiting 12:05:45 (7416): No heartbeat from core client for 30 sec - exiting 12:05:46 (7416): No heartbeat from core client for 30 sec - exiting 12:05:47 (7416): No heartbeat from core client for 30 sec - exiting 12:05:48 (7416): No heartbeat from core client for 30 sec - exiting 12:05:49 (7416): No heartbeat from core client for 30 sec - exiting 12:05:50 (7416): No heartbeat from core client for 30 sec - exiting 12:05:51 (7416): No heartbeat from core client for 30 sec - exiting 12:05:52 (7416): No heartbeat from core client for 30 sec - exiting 12:05:53 (7416): No heartbeat from core client for 30 sec - exiting 12:05:54 (7416): No heartbeat from core client for 30 sec - exiting 12:05:55 (7416): No heartbeat from core client for 30 sec - exiting 12:05:56 (7416): No heartbeat from core client for 30 sec - exiting 12:05:57 (7416): No heartbeat from core client for 30 sec - exiting 12:05:58 (7416): No heartbeat from core client for 30 sec - exiting 12:05:59 (7416): No heartbeat from core client for 30 sec - exiting 12:06:01 (7416): No heartbeat from core client for 30 sec - exiting 12:06:29 (7416): No heartbeat from core client for 30 sec - exiting 12:06:30 (7416): No heartbeat from core client for 30 sec - exiting 12:06:31 (7416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:06:32 (7416): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8144, selfPID=5868, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1580, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5428, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5312, selfPID=6372, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6604, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 8 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4444, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 8 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6828, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6896, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 8 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6356, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 9 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 9 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3744, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1932, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 10 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1240, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8128, selfPID=6872, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7408, selfPID=4988, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 11 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7692, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5804, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2456, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Mar 2012 08:46:13	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	138,336	721,184	5.2133
20 Mar 2012 13:13:24	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	126,816	661,830	5.2188
15 Mar 2012 14:18:12	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	115,296	601,896	5.2204
12 Mar 2012 14:09:03	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	103,776	541,598	5.2189
07 Mar 2012 15:32:29	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	92,256	481,404	5.2181
02 Mar 2012 18:24:51	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	80,736	421,736	5.2236
28 Feb 2012 10:13:06	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	69,216	362,060	5.2309
23 Feb 2012 14:32:12	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	57,696	301,146	5.2195
21 Feb 2012 10:13:48	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	46,179	241,337	5.2261
20 Feb 2012 18:55:33	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	46,176	240,632	5.2112
16 Feb 2012 12:58:29	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	34,656	179,889	5.1907
13 Feb 2012 18:30:18	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	23,136	120,426	5.2051
09 Feb 2012 14:01:34	945967	13979321	hadam3p_pnw_8sm0_2003_1_007710036_0	11,616	61,517	5.2959