Task 11879814

Name	hadam3p_pnw_v3ef_1999_1_006724450_0
Workunit	6927700
Created	10 Sep 2010, 8:11:39 UTC
Sent	13 Sep 2010, 10:42:33 UTC
Report deadline	26 Aug 2011, 16:02:33 UTC
Received	1 Oct 2010, 13:14:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1097721
Run time	4 days 23 hours 47 min 53 sec
CPU time	4 days 20 hours 28 min 27 sec
Validate state	Invalid
Credit	2,505.24
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.05 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> 14:02:21 (3416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5988, selfPID=5988, iMonCtr=2 17:37:34 (404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:47 (5180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:50 (5180): No heartbeat from core client for 30 sec - exiting 19:08:53 (5180): No heartbeat from core client for 30 sec - exiting 19:08:56 (5180): No heartbeat from core client for 30 sec - exiting 19:08:59 (5180): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6120, iMonCtr=2 Model crash detected, will try to restart... 11:57:42 (5092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:57:43 (5092): No heartbeat from core client for 30 sec - exiting R1:57:45 (5092): No heartbeat from core client foional Wor - exitingk er:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5536, sel11:58:47 (4332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5284, selfPID=5284, iMonCtr=2 12:16:47 (2076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:16:49 (2076): No heartbeat from core client for 30 sec - exiting 12:16:50 (2076): No heartbeat from core client for 30 sec - exiting 12:18:55 (4488): No heartbeat from core client for 30 sec - exiting 12:18:57 (4488): No heartbeat from core client for 30 sec - exiting 12:18:58 (4488): No heartbeat from core client for 30 sec - exiting 12:18:59 (4488): No heartbeat from core client for 30 sec - exiting 12:19:00 (4488): No heartbeat from core client for 30 sec - exiting 12:19:01 (4488): No heartbeat from core client for 30 sec - exiting 12:19:03 (4488): No heartbeat from core client for 30 sec - exiting 12:19:04 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:19:45 (2816): Can't acquire lockfile (32) - waiting 35s Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:36:36 (2816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:41:50 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5560, selfPID=5560, iMonCtr=2 20:31:08 (5028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:31:12 (5028): No heartbeat from core client for 30 sec - exiting 20:31:15 (5028): No heartbeat from core client for 30 sec - exiting 20:36:19 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:03:51 (2296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:03:53 (2296): No heartbeat from core client for 30 sec - exiting 14:03:55 (2296): No heartbeat from core client for 30 sec - exiting 14:55:36 (6012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6876, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6472, selfPID=5296, iMonCtr=1 Model crash detected, will try to restart... CCSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:29:29 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:29:30 (4284): No heartbeat from core client for 30 sec - exiting 12:09:31 (1704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:19:37 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:19:38 (4748): No heartbeat from core client for 30 sec - exiting 13:19:41 (4748): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:21:48 (5332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:21:50 (5332): No heartbeat from core client for 30 sec - exiting 22:21:52 (5332): No heartbeat from core client for 30 sec - exiting 22:26:48 (5396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:48 (5596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:50 (5596): No heartbeat from core client for 30 sec - exiting 22:34:12 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:34:14 (6112): No heartbeat from core client for 30 sec - exiting 22:34:16 (6112): No heartbeat from core client for 30 sec - exiting 22:39:35 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:39:37 (6356): No heartbeat from core client for 30 sec - exiting 22:39:39 (6356): No heartbeat from core client for 30 sec - exiting checkPID=2976, selfPID=2976, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:18:15 (3948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:18:17 (3948): No heartbeat from core client for 30 sec - exiting R1gion:1 8:2ker (3948): po hearteseat from core client for 30 sec - exiting 11:33:54 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... GSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:51:02 (3152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:03 (3152): No heartbeat from core client for 30 sec - exiting R6:51:05 (3152): No heartbeat from core client for 30 sec - exiting egional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3516, selfPID=3516, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... 18:22:36 (4988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:34:57 (1348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:34:58 (1348): No heartbeat from core client for 30 sec - exiting 18:34:59 (1348): No heartbeat from core client for 30 sec - exiting 18:40:04 (5152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:40:06 (5152): No heartbeat from core client for 30 sec - exiting 1egional Worker:: CPDN process is not runninf, exitirgom ,ore blient for 30 sec - exitetVal = 1, ng checkPID=3408, selfPI18:42:24 (6532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=2 14:01:36 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:41 (5612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:50 (5376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:52 (5376): No heartbeat from core client for 30 sec - exiting 17:36:53 (5376): No heartbeat from core client for 30 sec - exiting 13:07:28 (3120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:07:30 (3120): No heartbeat from core client for 30 sec - exiting 13:07:33 (3120): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=4376, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2660, selfPID=4572, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 10 13:08:55 (4572): called boinc_finish 14:18:11 (4508): No heartbeat from core client for 30 sec - exiting 14:18:12 (4508): No heartbeat from core client for 30 sec - exiting 14:18:13 (4508): No heartbeat from core client for 30 sec - exiting 14:18:15 (4508): No heartbeat from core client for 30 sec - exiting 14:18:16 (4508): No heartbeat from core client for 30 sec - exiting 14:18:17 (4508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_pnw_v3ef_1999_1_006724450_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_v3ef_1999_1_006724450_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Sep 2010 10:17:01	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	115,296	392,820	3.4071
27 Sep 2010 15:50:53	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	103,776	352,330	3.3951
25 Sep 2010 09:11:44	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	92,256	308,715	3.3463
23 Sep 2010 17:21:38	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	80,736	268,269	3.3228
22 Sep 2010 11:33:23	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	69,216	229,990	3.3228
21 Sep 2010 07:25:04	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	57,696	192,643	3.3389
19 Sep 2010 15:20:46	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	46,176	153,881	3.3325
17 Sep 2010 09:25:17	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	34,656	115,825	3.3421
16 Sep 2010 08:29:53	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	23,136	78,351	3.3865
14 Sep 2010 14:43:43	1097721	11879814	hadam3p_pnw_v3ef_1999_1_006724450_0	11,616	40,138	3.4554