Task 16443583

Name	hadam3p_anz_a0u5_2012_1_008612605_0
Workunit	8759117
Created	2 Apr 2014, 14:24:51 UTC
Sent	1 May 2014, 17:29:16 UTC
Report deadline	13 Apr 2015, 22:49:16 UTC
Received	6 Jul 2014, 23:01:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1143523
Run time	8 days 0 hours 20 min 13 sec
CPU time	7 days 8 hours 48 min 41 sec
Validate state	Invalid
Credit	3,490.64
Device peak FLOPS	2.55 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:08:36 (4708): No heartbeat from core client for 30 sec - exiting 14:08:37 (4708): No heartbeat from core client for 30 sec - exiting 14:08:38 (4708): No heartbeat from core client for 30 sec - exiting 14:08:39 (4708): No heartbeat from core client for 30 sec - exiting 14:08:40 (4708): No heartbeat from core client for 30 sec - exiting 14:08:41 (4708): No heartbeat from core client for 30 sec - exiting 14:08:42 (4708): No heartbeat from core client for 30 sec - exiting 14:08:43 (4708): No heartbeat from core client for 30 sec - exiting 14:08:44 (4708): No heartbeat from core client for 30 sec - exiting 14:08:45 (4708): No heartbeat from core client for 30 sec - exiting 14:08:46 (4708): No heartbeat from core client for 30 sec - exiting 14:08:47 (4708): No heartbeat from core client for 30 sec - exiting 14:08:48 (4708): No heartbeat from core client for 30 sec - exiting 14:08:49 (4708): No heartbeat from core client for 30 sec - exiting 14:08:50 (4708): No heartbeat from core client for 30 sec - exiting 14:08:51 (4708): No heartbeat from core client for 30 sec - exiting 14:08:52 (4708): No heartbeat from core client for 30 sec - exiting 14:08:53 (4708): No heartbeat from core client for 30 sec - exiting 14:08:54 (4708): No heartbeat from core client for 30 sec - exiting 14:08:55 (4708): No heartbeat from core client for 30 sec - exiting 14:08:56 (4708): No heartbeat from core client for 30 sec - exiting 14:08:57 (4708): No heartbeat from core client for 30 sec - exiting 14:08:58 (4708): No heartbeat from core client for 30 sec - exiting 14:08:59 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4768, selfPID=4768, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:26:51 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3940, selfPID=3180, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3460, selfPID=5688, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5452, selfPID=3772, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15440, selfPID=15440, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17228, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4272, selfPID=4272, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3584, selfPID=6060, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5164, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3316, selfPID=5900, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4580, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4780, selfPID=3776, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:07:23 (5820): No heartbeat from core client for 30 sec - exiting 19:07:24 (5820): No heartbeat from core client for 30 sec - exiting 19:07:25 (5820): No heartbeat from core client for 30 sec - exiting 19:07:26 (5820): No heartbeat from core client for 30 sec - exiting 19:07:27 (5820): No heartbeat from core client for 30 sec - exiting 19:07:28 (5820): No heartbeat from core client for 30 sec - exiting 19:07:29 (5820): No heartbeat from core client for 30 sec - exiting 19:07:30 (5820): No heartbeat from core client for 30 sec - exiting 19:07:31 (5820): No heartbeat from core client for 30 sec - exiting 19:07:32 (5820): No heartbeat from core client for 30 sec - exiting 19:07:33 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:14:51 (5192): No heartbeat from core client for 30 sec - exiting 17:14:52 (5192): No heartbeat from core client for 30 sec - exiting 17:14:53 (5192): No heartbeat from core client for 30 sec - exiting 17:14:54 (5192): No heartbeat from core client for 30 sec - exiting 17:14:55 (5192): No heartbeat from core client for 30 sec - exiting 17:14:56 (5192): No heartbeat from core client for 30 sec - exiting 17:14:57 (5192): No heartbeat from core client for 30 sec - exiting 17:14:58 (5192): No heartbeat from core client for 30 sec - exiting 17:14:59 (5192): No heartbeat from core client for 30 sec - exiting 17:15:00 (5192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6108, selfPID=6108, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1256, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5684, selfPID=5684, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:28:59 (5832): No heartbeat from core client for 30 sec - exiting 09:29:00 (5832): No heartbeat from core client for 30 sec - exiting 09:29:01 (5832): No heartbeat from core client for 30 sec - exiting 09:29:02 (5832): No heartbeat from core client for 30 sec - exiting 09:29:03 (5832): No heartbeat from core client for 30 sec - exiting 09:29:04 (5832): No heartbeat from core client for 30 sec - exiting 09:29:05 (5832): No heartbeat from core client for 30 sec - exiting 09:29:06 (5832): No heartbeat from core client for 30 sec - exiting 09:29:07 (5832): No heartbeat from core client for 30 sec - exiting 09:29:08 (5832): No heartbeat from core client for 30 sec - exiting 09:29:09 (5832): No heartbeat from core client for 30 sec - exiting 09:29:10 (5832): No heartbeat from core client for 30 sec - exiting 09:29:11 (5832): No heartbeat from core client for 30 sec - exiting 09:29:12 (5832): No heartbeat from core client for 30 sec - exiting 09:29:13 (5832): No heartbeat from core client for 30 sec - exiting 09:29:14 (5832): No heartbeat from core client for 30 sec - exiting 09:29:15 (5832): No heartbeat from core client for 30 sec - exiting 09:29:16 (5832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5400, selfPID=5400, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7604, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5920, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5184, selfPID=5604, iMonCtr=1 Model crash detected, will try to restart... 18:39:17 (4164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1152, selfPID=1152, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5796, selfPID=5704, iMonCtr=1 Model crash detected, will try to restart... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 Leaving CPDN_Main::Monitor... zip error: Could not create output file (was replacing the original zip file) Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_a0u5_2012_1_008612605_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a0u5_2012_1_008612605_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a0u5_2012_1_008612605_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a0u5_2012_1_008612605_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a0u5_2012_1_008612605_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jun 2014 23:38:59	1143523	16443583	hadam3p_anz_a0u5_2012_1_008612605_0	80,939	577,902	7.1400
27 Jun 2014 01:24:47	1143523	16443583	hadam3p_anz_a0u5_2012_1_008612605_0	69,419	500,613	7.2115
23 Jun 2014 17:34:45	1143523	16443583	hadam3p_anz_a0u5_2012_1_008612605_0	57,899	419,621	7.2475
11 Jun 2014 23:08:32	1143523	16443583	hadam3p_anz_a0u5_2012_1_008612605_0	46,379	336,436	7.2541
01 Jun 2014 15:15:49	1143523	16443583	hadam3p_anz_a0u5_2012_1_008612605_0	34,859	252,947	7.2563
22 May 2014 13:38:16	1143523	16443583	hadam3p_anz_a0u5_2012_1_008612605_0	23,339	170,391	7.3007
10 May 2014 21:38:45	1143523	16443583	hadam3p_anz_a0u5_2012_1_008612605_0	11,819	86,349	7.3059