Task 14445980

Name	hadam3p_pnw_bg53_1998_1_007907222_0
Workunit	8062334
Created	17 Apr 2012, 18:32:25 UTC
Sent	11 May 2012, 14:27:09 UTC
Report deadline	23 Apr 2013, 19:47:09 UTC
Received	3 Aug 2012, 3:49:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1204185
Run time	7 days 16 hours 21 min 2 sec
CPU time	6 hours 39 min 44 sec
Validate state	Invalid
Credit	1,504.00
Device peak FLOPS	1.36 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3052, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4424, selfPID=4424, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5976, selfPID=5976, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 14:59:07 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:14:11 (3780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:14:15 (3780): No heartbeat from core client for 30 sec - exiting 15:29:17 (2800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:44:48 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:44:55 (5048): No heartbeat from core client for 30 sec - exiting 15:44:56 (5048): No heartbeat from core client for 30 sec - exiting 15:44:57 (5048): No heartbeat from core client for 30 sec - exiting 15:44:59 (5048): No heartbeat from core client for 30 sec - exiting 15:45:00 (5048): No heartbeat from core client for 30 sec - exiting 15:45:01 (5048): No heartbeat from core client for 30 sec - exiting 16:02:30 (4592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:02:36 (4592): No heartbeat from core client for 30 sec - exiting 16:02:37 (4592): No heartbeat from core client for 30 sec - exiting 16:02:38 (4592): No heartbeat from core client for 30 sec - exiting 16:02:40 (4592): No heartbeat from core client for 30 sec - exiting 16:02:41 (4592): No heartbeat from core client for 30 sec - exiting 16:02:42 (4592): No heartbeat from core client for 30 sec - exiting 16:02:43 (4592): No heartbeat from core client for 30 sec - exiting 16:02:45 (4592): No heartbeat from core client for 30 sec - exiting 16:02:46 (4592): No heartbeat from core client for 30 sec - exiting 16:02:47 (4592): No heartbeat from core client for 30 sec - exiting 16:02:48 (4592): No heartbeat from core client for 30 sec - exiting 16:02:49 (4592): No heartbeat from core client for 30 sec - exiting 16:22:19 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:27 (5112): No heartbeat from core client for 30 sec - exiting 16:22:28 (5112): No heartbeat from core client for 30 sec - exiting 16:22:30 (5112): No heartbeat from core client for 30 sec - exiting 16:22:31 (5112): No heartbeat from core client for 30 sec - exiting 16:22:33 (5112): No heartbeat from core client for 30 sec - exiting 16:22:34 (5112): No heartbeat from core client for 30 sec - exiting 16:22:35 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:43:41 (5848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:43:47 (5848): No heartbeat from core client for 30 sec - exiting 01:43:48 (5848): No heartbeat from core client for 30 sec - exiting 01:43:49 (5848): No heartbeat from core client for 30 sec - exiting 01:43:50 (5848): No heartbeat from core client for 30 sec - exiting 01:43:51 (5848): No heartbeat from core client for 30 sec - exiting 01:43:52 (5848): No heartbeat from core client for 30 sec - exiting 01:43:54 (5848): No heartbeat from core client for 30 sec - exiting 01:43:55 (5848): No heartbeat from core client for 30 sec - exiting 01:43:56 (5848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2228, selfPID=2228, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 05:35:18 (5392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:50:55 (2504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6128, selfPID=6128, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6128, selfPID=4616, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 6 04:18:11 (4616): No heartbeat from core client for 30 sec - exiting 04:18:13 (4616): No heartbeat from core client for 30 sec - exiting 04:18:14 (4616): No heartbeat from core client for 30 sec - exiting 04:18:15 (4616): No heartbeat from core client for 30 sec - exiting 04:18:16 (4616): No heartbeat from core client for 30 sec - exiting 04:18:17 (4616): No heartbeat from core client for 30 sec - exiting 04:18:18 (4616): No heartbeat from core client for 30 sec - exiting 04:18:19 (4616): No heartbeat from core client for 30 sec - exiting 04:18:20 (4616): No heartbeat from core client for 30 sec - exiting 04:18:21 (4616): No heartbeat from core client for 30 sec - exiting 04:18:22 (4468): Can't acquire lockfile (32) - waiting 35s 04:18:22 (4616): No heartbeat from core client for 30 sec - exiting 04:18:23 (4616): No heartbeat from core client for 30 sec - exiting 04:18:24 (4616): No heartbeat from core client for 30 sec - exiting 04:18:25 (4616): No heartbeat from core client for 30 sec - exiting 04:18:26 (4616): No heartbeat from core client for 30 sec - exiting 04:18:27 (4616): No heartbeat from core client for 30 sec - exiting 04:18:28 (4616): No heartbeat from core client for 30 sec - exiting 04:18:29 (4616): No heartbeat from core client for 30 sec - exiting 04:18:30 (4616): No heartbeat from core client for 30 sec - exiting 04:18:31 (4616): No heartbeat from core client for 30 sec - exiting 04:18:32 (4616): No heartbeat from core client for 30 sec - exiting 04:18:33 (4616): No heartbeat from core client for 30 sec - exiting 04:18:34 (4616): No heartbeat from core client for 30 sec - exiting 04:18:35 (4616): No heartbeat from core client for 30 sec - exiting 04:18:36 (4616): No heartbeat from core client for 30 sec - exiting 04:18:37 (4616): No heartbeat from core client for 30 sec - exiting 04:18:38 (4616): No heartbeat from core client for 30 sec - exiting 04:18:39 (4616): No heartbeat from core client for 30 sec - exiting 04:18:40 (4616): No heartbeat from core client for 30 sec - exiting 04:18:41 (4616): No heartbeat from core client for 30 sec - exiting 04:18:42 (4616): No heartbeat from core client for 30 sec - exiting 04:18:43 (4616): No heartbeat from core client for 30 sec - exiting 04:18:44 (4616): No heartbeat from core client for 30 sec - exiting 04:18:45 (4616): No heartbeat from core client for 30 sec - exiting 04:18:46 (4616): No heartbeat from core client for 30 sec - exiting 04:18:47 (4616): No heartbeat from core client for 30 sec - exiting 04:18:48 (4616): No heartbeat from core client for 30 sec - exiting 04:18:49 (4616): No heartbeat from core client for 30 sec - exiting 04:18:50 (4616): No heartbeat from core client for 30 sec - exiting 04:18:51 (4616): No heartbeat from core client for 30 sec - exiting 04:18:52 (4616): No heartbeat from core client for 30 sec - exiting 04:18:53 (4616): No heartbeat from core client for 30 sec - exiting 04:18:57 (4468): Can't acquire lockfile (32) - exiting 04:18:57 (4468): Error: The process cannot access the file because it is being used by another process. (0x20) Called boinc_finish 04:18:59 (4616): No heartbeat from core client for 30 sec - exiting 04:19:00 (4616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4208, selfPID=4208, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4208, selfPID=2148, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 0 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_bg53_1998_1_007907222_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg53_1998_1_007907222_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg53_1998_1_007907222_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg53_1998_1_007907222_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg53_1998_1_007907222_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_bg53_1998_1_007907222_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Jul 2012 10:04:20	1204185	14445980	hadam3p_pnw_bg53_1998_1_007907222_0	69,217	487,807	7.0475
28 Jul 2012 09:04:10	1204185	14445980	hadam3p_pnw_bg53_1998_1_007907222_0	69,216	486,777	7.0327
26 Jul 2012 13:26:26	1204185	14445980	hadam3p_pnw_bg53_1998_1_007907222_0	57,696	407,090	7.0558
24 Jul 2012 05:33:19	1204185	14445980	hadam3p_pnw_bg53_1998_1_007907222_0	46,176	327,049	7.0827
22 Jul 2012 23:30:31	1204185	14445980	hadam3p_pnw_bg53_1998_1_007907222_0	34,656	246,122	7.1019
21 Jul 2012 13:12:06	1204185	14445980	hadam3p_pnw_bg53_1998_1_007907222_0	23,136	161,171	6.9662
20 Jul 2012 01:40:41	1204185	14445980	hadam3p_pnw_bg53_1998_1_007907222_0	11,616	78,255	6.7368