Task 17813664

Name	hadam3p_anz_e2ez_2012_1_009293632_1
Workunit	9377820
Created	21 Jan 2015, 19:44:33 UTC
Sent	21 Jan 2015, 19:45:53 UTC
Report deadline	4 Jan 2016, 1:05:53 UTC
Received	3 Mar 2015, 11:20:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1340045
Run time	2 days 1 hours 56 min 50 sec
CPU time	8 hours 44 min 15 sec
Validate state	Invalid
Credit	509.72
Device peak FLOPS	3.26 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.27</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3972, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2044, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5040, selfPID=4808, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5948, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2248, selfPID=4292, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4540, iMonCtr=2 CPDN Monitor - Quit request from BOINC... GCController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4420, selfPID=3972, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=816, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3672, selfPID=4660, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6120, selfPID=4220, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5820, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1132, selfPID=4848, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1568, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3576, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2796, selfPID=4264, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=712, selfPID=2948, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3196, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4168, selfPID=3916, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3180, selfPID=4008, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3800, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3620, iMonCtr=2 Model crash detected, will try to restart... GCoobal Workerntroller:: CPDN prrocess iscess is no not running, eing, bRetVal = 1, checkPID=0, selfPID=5112, iMonCtr=2 Model crash detected, will try to restart... xiting, bRetVal = 1, checkPID=0, selfPID=3748, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4768, selfPID=4696, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3492, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6472, selfPID=5756, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2320, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2536, selfPID=2824, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2332, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4080, iMonCtr=2 Model crash detected, will try to restart... 19:04:40 (3200): No heartbeat from core client for 30 sec - exiting 19:04:41 (3200): No heartbeat from core client for 30 sec - exiting 19:04:42 (3200): No heartbeat from core client for 30 sec - exiting 19:04:44 (3200): No heartbeat from core client for 30 sec - exiting 19:04:45 (3200): No heartbeat from core client for 30 sec - exiting 19:04:46 (3200): No heartbeat from core client for 30 sec - exiting 19:04:47 (3200): No heartbeat from core client for 30 sec - exiting 19:04:48 (3200): No heartbeat from core client for 30 sec - exiting 19:04:49 (3200): No heartbeat from core client for 30 sec - exiting 19:04:50 (3200): No heartbeat from core client for 30 sec - exiting 19:04:51 (3200): No heartbeat from core client for 30 sec - exiting 19:04:52 (3200): No heartbeat from core client for 30 sec - exiting 19:04:53 (3200): No heartbeat from core client for 30 sec - exiting 19:04:55 (3200): No heartbeat from core client for 30 sec - exiting 19:04:56 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4132, selfPID=3492, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2452, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3880, selfPID=1452, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5948, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6012, selfPID=4132, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5352, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5420, selfPID=4932, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4300, iMonCtr=2 Model crash detected, will try to restart... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5624, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4140, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3100, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4464, selfPID=5012, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3852, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3900, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6136, selfPID=3816, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1332, selfPID=2856, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1852, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1020, selfPID=4536, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4328, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5776, selfPID=4568, iMonCtr=1 Model crash detected, will try to restart... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4716, selfPID=3968, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4352, selfPID=4904, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2328, iMonCtr=2 Model crash detected, will try to restart... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1352, selfPID=1352, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1352, selfPID=752, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_2.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_e2ez_2012_1_009293632_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Feb 2015 18:17:32	1340045	17813664	hadam3p_anz_e2ez_2012_1_009293632_1	11,819	7,408	0.6268