Task 14575963

Name	hadam3p_pnw_ys51_1996_1_006881933_1
Workunit	7085249
Created	23 Apr 2012, 11:04:18 UTC
Sent	24 Apr 2012, 22:04:01 UTC
Report deadline	7 Apr 2013, 3:24:01 UTC
Received	5 Nov 2012, 8:17:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1214410
Run time	2 days 11 hours 57 min 52 sec
CPU time	2 days 4 hours 25 min 43 sec
Validate state	Invalid
Credit	1,003.35
Device peak FLOPS	3.00 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1184, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5024, selfPID=5024, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2024, selfPID=2024, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4588, selfPID=4588, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:47:00 (3760): No heartbeat from core client for 30 sec - exiting 20:47:01 (3760): No heartbeat from core client for 30 sec - exiting 20:47:02 (3760): No heartbeat from core client for 30 sec - exiting 20:47:03 (3760): No heartbeat from core client for 30 sec - exiting 20:47:04 (3760): No heartbeat from core client for 30 sec - exiting 20:47:06 (3760): No heartbeat from core client for 30 sec - exiting 20:47:07 (3760): No heartbeat from core client for 30 sec - exiting 20:47:08 (3760): No heartbeat from core client for 30 sec - exiting 20:47:09 (3760): No heartbeat from core client for 30 sec - exiting 20:47:10 (3760): No heartbeat from core client for 30 sec - exiting 20:47:11 (3760): No heartbeat from core client for 30 sec - exiting 20:47:12 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:20:20 (3624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1180, selfPID=1180, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1272, selfPID=1272, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:06:11 (1504): No heartbeat from core client for 30 sec - exiting 16:06:12 (1504): No heartbeat from core client for 30 sec - exiting 16:06:13 (1504): No heartbeat from core client for 30 sec - exiting 16:06:14 (1504): No heartbeat from core client for 30 sec - exiting 16:06:15 (1504): No heartbeat from core client for 30 sec - exiting 16:06:16 (1504): No heartbeat from core client for 30 sec - exiting 16:06:17 (1504): No heartbeat from core client for 30 sec - exiting 16:06:18 (1504): No heartbeat from core client for 30 sec - exiting 16:06:19 (1504): No heartbeat from core client for 30 sec - exiting 16:06:20 (1504): No heartbeat from core client for 30 sec - exiting 16:06:21 (1504): No heartbeat from core client for 30 sec - exiting 16:06:22 (1504): No heartbeat from core client for 30 sec - exiting 16:06:24 (1504): No heartbeat from core client for 30 sec - exiting 16:06:25 (1504): No heartbeat from core client for 30 sec - exiting 16:06:26 (1504): No heartbeat from core client for 30 sec - exiting 16:06:27 (1504): No heartbeat from core client for 30 sec - exiting 16:06:28 (1504): No heartbeat from core client for 30 sec - exiting 16:06:29 (1504): No heartbeat from core client for 30 sec - exiting 16:06:30 (1504): No heartbeat from core client for 30 sec - exiting 16:06:31 (1504): No heartbeat from core client for 30 sec - exiting 16:06:32 (1504): No heartbeat from core client for 30 sec - exiting 16:06:33 (1504): No heartbeat from core client for 30 sec - exiting 16:06:34 (1504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=984, selfPID=984, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:22:11 (2960): No heartbeat from core client for 30 sec - exiting 20:22:13 (2960): No heartbeat from core client for 30 sec - exiting 20:22:14 (2960): No heartbeat from core client for 30 sec - exiting 20:22:15 (2960): No heartbeat from core client for 30 sec - exiting 20:22:16 (2960): No heartbeat from core client for 30 sec - exiting 20:22:17 (2960): No heartbeat from core client for 30 sec - exiting 20:22:18 (2960): No heartbeat from core client for 30 sec - exiting 20:22:19 (2960): No heartbeat from core client for 30 sec - exiting 20:22:20 (2960): No heartbeat from core client for 30 sec - exiting 20:22:21 (2960): No heartbeat from core client for 30 sec - exiting 20:22:22 (2960): No heartbeat from core client for 30 sec - exiting 20:22:23 (2960): No heartbeat from core client for 30 sec - exiting 20:22:24 (2960): No heartbeat from core client for 30 sec - exiting 20:22:26 (2960): No heartbeat from core client for 30 sec - exiting 20:22:27 (2960): No heartbeat from core client for 30 sec - exiting 20:22:28 (2960): No heartbeat from core client for 30 sec - exiting 20:22:29 (2960): No heartbeat from core client for 30 sec - exiting 20:22:30 (2960): No heartbeat from core client for 30 sec - exiting 20:22:31 (2960): No heartbeat from core client for 30 sec - exiting 20:22:32 (2960): No heartbeat from core client for 30 sec - exiting 20:22:33 (2960): No heartbeat from core client for 30 sec - exiting 20:22:34 (2960): No heartbeat from core client for 30 sec - exiting 20:22:35 (2960): No heartbeat from core client for 30 sec - exiting 20:22:36 (2960): No heartbeat from core client for 30 sec - exiting 20:22:38 (2960): No heartbeat from core client for 30 sec - exiting 20:22:39 (2960): No heartbeat from core client for 30 sec - exiting 20:22:40 (2960): No heartbeat from core client for 30 sec - exiting 20:22:41 (2960): No heartbeat from core client for 30 sec - exiting 20:22:42 (2960): No heartbeat from core client for 30 sec - exiting 20:22:43 (2960): No heartbeat from core client for 30 sec - exiting 20:22:44 (2960): No heartbeat from core client for 30 sec - exiting 20:22:45 (2960): No heartbeat from core client for 30 sec - exiting 20:22:46 (2960): No heartbeat from core client for 30 sec - exiting 20:22:47 (2960): No heartbeat from core client for 30 sec - exiting 20:22:49 (2960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4872, selfPID=4872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1488, selfPID=1488, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1260, selfPID=1260, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1596, selfPID=1596, iMonCtr=2 CCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4264, selfPID=4264, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3120, selfPID=3120, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3120, selfPID=2152, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 4 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_ys51_1996_1_006881933_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Jun 2012 09:52:47	1214410	14575963	hadam3p_pnw_ys51_1996_1_006881933_1	46,176	152,270	3.2976
05 Jun 2012 23:44:53	1214410	14575963	hadam3p_pnw_ys51_1996_1_006881933_1	34,656	115,338	3.3281
24 May 2012 21:58:38	1214410	14575963	hadam3p_pnw_ys51_1996_1_006881933_1	23,136	74,620	3.2253
28 Apr 2012 00:35:54	1214410	14575963	hadam3p_pnw_ys51_1996_1_006881933_1	11,616	36,108	3.1085