Task 17593139

Name	hadam3p_anz_m5og_2012_1_009306886_0
Workunit	9391074
Created	17 Dec 2014, 19:50:56 UTC
Sent	21 Dec 2014, 23:38:59 UTC
Report deadline	4 Dec 2015, 4:58:59 UTC
Received	10 Jan 2015, 13:48:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1143523
Run time	2 days 16 hours 24 min 43 sec
CPU time	2 days 12 hours 17 min
Validate state	Invalid
Credit	1,006.54
Device peak FLOPS	2.89 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <stderr_txt> 22:19:14 (5356): No heartbeat from core client for 30 sec - exiting 22:19:15 (5356): No heartbeat from core client for 30 sec - exiting 22:19:16 (5356): No heartbeat from core client for 30 sec - exiting 22:19:17 (5356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3988, iMonCtr=2 13:00:12 (6052): No heartbeat from core client for 30 sec - exiting 13:00:13 (6052): No heartbeat from core client for 30 sec - exiting 13:00:14 (6052): No heartbeat from core client for 30 sec - exiting 13:00:15 (6052): No heartbeat from core client for 30 sec - exiting 13:00:16 (6052): No heartbeat from core client for 30 sec - exiting 13:00:17 (6052): No heartbeat from core client for 30 sec - exiting 13:00:18 (6052): No heartbeat from core client for 30 sec - exiting 13:00:19 (6052): No heartbeat from core client for 30 sec - exiting 13:00:20 (6052): No heartbeat from core client for 30 sec - exiting 13:00:21 (6052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:42:22 (5788): No heartbeat from core client for 30 sec - exiting 19:42:23 (5788): No heartbeat from core client for 30 sec - exiting 19:42:24 (5788): No heartbeat from core client for 30 sec - exiting 19:42:25 (5788): No heartbeat from core client for 30 sec - exiting 19:42:26 (5788): No heartbeat from core client for 30 sec - exiting 19:42:27 (5788): No heartbeat from core client for 30 sec - exiting 19:42:28 (5788): No heartbeat from core client for 30 sec - exiting 19:42:29 (5788): No heartbeat from core client for 30 sec - exiting 19:42:30 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:41 (4628): No heartbeat from core client for 30 sec - exiting 13:49:42 (4628): No heartbeat from core client for 30 sec - exiting 13:49:43 (4628): No heartbeat from core client for 30 sec - exiting 13:49:44 (4628): No heartbeat from core client for 30 sec - exiting 13:49:45 (4628): No heartbeat from core client for 30 sec - exiting 13:49:46 (4628): No heartbeat from core client for 30 sec - exiting 13:49:47 (4628): No heartbeat from core client for 30 sec - exiting 13:49:48 (4628): No heartbeat from core client for 30 sec - exiting 13:49:49 (4628): No heartbeat from core client for 30 sec - exiting 13:49:50 (4628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN proces17:58:42 (4596): No heartbeat from core client for 30 sec - exiting 17:58:43 (4596): No heartbeat from core client for 30 sec - exiting 17:58:44 (4596): No heartbeat from core client for 30 sec - exiting 17:58:45 (4596): No heartbeat from core client for 30 sec - exiting 17:58:46 (4596): No heartbeat from core client for 30 sec - exiting 17:58:47 (4596): No heartbeat from core client for 30 sec - exiting 17:58:48 (4596): No heartbeat from core client for 30 sec - exiting 17:58:49 (4596): No heartbeat from core client for 30 sec - exiting 17:58:50 (4596): No heartbeat from core client for 30 sec - exiting 17:58:51 (4596): No heartbeat from core client for 30 sec - exiting 17:58:52 (4596): No heartbeat from core client for 30 sec - exiting 17:58:53 (4596): No heartbeat from core client for 30 sec - exiting 17:58:54 (4596): No heartbeat from core client for 30 sec - exiting 17:58:55 (4596): No heartbeat from core client for 30 sec - exiting 17:58:56 (4596): No heartbeat from core client for 30 sec - exiting 17:58:57 (4596): No heartbeat from core client for 30 sec - exiting 17:58:58 (4596): No heartbeat from core client for 30 sec - exiting 17:58:59 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:42 (5696): No heartbeat from core client for 30 sec - exiting 22:26:43 (5696): No heartbeat from core client for 30 sec - exiting 22:26:44 (5696): No heartbeat from core client for 30 sec - exiting 22:26:45 (5696): No heartbeat from core client for 30 sec - exiting 22:26:46 (5696): No heartbeat from core client for 30 sec - exiting 22:26:47 (5696): No heartbeat from core client for 30 sec - exiting 22:26:48 (5696): No heartbeat from core client for 30 sec - exiting 22:26:49 (5696): No heartbeat from core client for 30 sec - exiting 22:26:50 (5696): No heartbeat from core client for 30 sec - exiting 22:26:51 (5696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5596, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5744, selfPID=4348, iMonCtr=1 Model crash detected, will try to restart... 13:01:03 (5384): No heartbeat from core client for 30 sec - exiting 13:01:04 (5384): No heartbeat from core client for 30 sec - exiting 13:01:05 (5384): No heartbeat from core client for 30 sec - exiting 13:01:06 (5384): No heartbeat from core client for 30 sec - exiting 13:01:07 (5384): No heartbeat from core client for 30 sec - exiting 13:01:08 (5384): No heartbeat from core client for 30 sec - exiting 13:01:09 (5384): No heartbeat from core client for 30 sec - exiting 13:01:10 (5384): No heartbeat from core client for 30 sec - exiting 13:01:11 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5548, iMonCtr=2 Model crash detected, will try to restart... 20:40:22 (4232): No heartbeat from core client for 30 sec - exiting 20:40:23 (4232): No heartbeat from core client for 30 sec - exiting 20:40:24 (4232): No heartbeat from core client for 30 sec - exiting 20:40:25 (4232): No heartbeat from core client for 30 sec - exiting 20:40:26 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:28 (5220): No heartbeat from core client for 30 sec - exiting 17:33:29 (5220): No heartbeat from core client for 30 sec - exiting 17:33:30 (5220): No heartbeat from core client for 30 sec - exiting 17:33:31 (5220): No heartbeat from core client for 30 sec - exiting 17:33:32 (5220): No heartbeat from core client for 30 sec - exiting 17:33:33 (5220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:03:03 (5440): No heartbeat from core client for 30 sec - exiting 18:03:04 (5440): No heartbeat from core client for 30 sec - exiting 18:03:05 (5440): No heartbeat from core client for 30 sec - exiting 18:03:06 (5440): No heartbeat from core client for 30 sec - exiting 18:03:07 (5440): No heartbeat from core client for 30 sec - exiting 18:03:08 (5440): No heartbeat from core client for 30 sec - exiting 18:03:09 (5440): No heartbeat from core client for 30 sec - exiting 18:03:10 (5440): No heartbeat from core client for 30 sec - exiting 18:03:11 (5440): No heartbeat from core client for 30 sec - exiting 18:03:12 (5440): No heartbeat from core client for 30 sec - exiting 18:03:13 (5440): No heartbeat from core client for 30 sec - exiting 18:03:14 (5440): No heartbeat from core client for 30 sec - exiting 18:03:15 (5440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:14 (5652): No heartbeat from core client for 30 sec - exiting 20:26:15 (5652): No heartbeat from core client for 30 sec - exiting 20:26:16 (5652): No heartbeat from core client for 30 sec - exiting 20:26:17 (5652): No heartbeat from core client for 30 sec - exiting 20:26:18 (5652): No heartbeat from core client for 30 sec - exiting 20:26:19 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:32:47 (4204): No heartbeat from core client for 30 sec - exiting 15:32:48 (4204): No heartbeat from core client for 30 sec - exiting 15:32:49 (4204): No heartbeat from core client for 30 sec - exiting 15:32:50 (4204): No heartbeat from core client for 30 sec - exiting 15:32:51 (4204): No heartbeat from core client for 30 sec - exiting 15:32:52 (4204): No heartbeat from core client for 30 sec - exiting 15:32:53 (4204): No heartbeat from core client for 30 sec - exiting 15:32:54 (4204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:56 (5524): No heartbeat from core client for 30 sec - exiting 17:40:57 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:53 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6092, selfPID=6092, iMonCtr=2 16:26:06 (5064): No heartbeat from core client for 30 sec - exiting 16:26:07 (5064): No heartbeat from core client for 30 sec - exiting 16:26:08 (5064): No heartbeat from core client for 30 sec - exiting 16:26:09 (5064): No heartbeat from core client for 30 sec - exiting 16:26:10 (5064): No heartbeat from core client for 30 sec - exiting 16:26:11 (5064): No heartbeat from core client for 30 sec - exiting 16:26:12 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5196, selfPID=5468, iMonCtr=1 Model crash detected, will try to restart... 17:38:35 (4556): No heartbeat from core client for 30 sec - exiting 17:38:36 (4556): No heartbeat from core client for 30 sec - exiting 17:38:37 (4556): No heartbeat from core client for 30 sec - exiting 17:38:38 (4556): No heartbeat from core client for 30 sec - exiting 17:38:39 (4556): No heartbeat from core client for 30 sec - exiting 17:38:40 (4556): No heartbeat from core client for 30 sec - exiting 17:38:41 (4556): No heartbeat from core client for 30 sec - exiting 17:38:42 (4556): No heartbeat from core client for 30 sec - exiting 17:38:43 (4556): No heartbeat from core client for 30 sec - exiting 17:38:44 (4556): No heartbeat from core client for 30 sec - exiting 17:38:45 (4556): No heartbeat from core client for 30 sec - exiting 17:38:46 (4556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5304, iMonCtr=2 22:03:37 (5532): No heartbeat from core client for 30 sec - exiting 22:03:38 (5532): No heartbeat from core client for 30 sec - exiting 22:03:39 (5532): No heartbeat from core client for 30 sec - exiting 22:03:40 (5532): No heartbeat from core client for 30 sec - exiting 22:03:41 (5532): No heartbeat from core client for 30 sec - exiting 22:03:42 (5532): No heartbeat from core client for 30 sec - exiting 22:03:43 (5532): No heartbeat from core client for 30 sec - exiting 22:03:44 (5532): No heartbeat from core client for 30 sec - exiting 22:03:45 (5532): No heartbeat from core client for 30 sec - exiting 22:03:46 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4276, selfPID=808, iMonCtr=1 Model crash detected, will try to restart... 11:41:24 (5216): No heartbeat from core client for 30 sec - exiting 11:41:25 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5200, selfPID=5500, iMonCtr=1 Model crash detected, will try to restart... 13:01:15 (5532): No heartbeat from core client for 30 sec - exiting 13:01:16 (5532): No heartbeat from core client for 30 sec - exiting 13:01:17 (5532): No heartbeat from core client for 30 sec - exiting 13:01:18 (5532): No heartbeat from core client for 30 sec - exiting 13:01:19 (5532): No heartbeat from core client for 30 sec - exiting 13:01:20 (5532): No heartbeat from core client for 30 sec - exiting 13:01:21 (5532): No heartbeat from core client for 30 sec - exiting 13:01:22 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:26:14 (5456): No heartbeat from core client for 30 sec - exiting 18:26:15 (5456): No heartbeat from core client for 30 sec - exiting 18:26:16 (5456): No heartbeat from core client for 30 sec - exiting 18:26:17 (5456): No heartbeat from core client for 30 sec - exiting 18:26:18 (5456): No heartbeat from core client for 30 sec - exiting 18:26:19 (5456): No heartbeat from core client for 30 sec - exiting 18:26:20 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5156, selfPID=5588, iMonCtr=1 Model crash detected, will try to restart... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1004, selfPID=5724, iMonCtr=1 Model crash detected, will try to restart... 20:07:30 (3096): No heartbeat from core client for 30 sec - exiting 20:07:31 (3096): No heartbeat from core client for 30 sec - exiting 20:07:32 (3096): No heartbeat from core client for 30 sec - exiting 20:07:33 (3096): No heartbeat from core client for 30 sec - exiting 20:07:34 (3096): No heartbeat from core client for 30 sec - exiting 20:07:35 (3096): No heartbeat from core client for 30 sec - exiting 20:07:36 (3096): No heartbeat from core client for 30 sec - exiting 20:07:37 (3096): No heartbeat from core client for 30 sec - exiting 20:07:38 (3096): No heartbeat from core client for 30 sec - exiting 20:07:39 (3096): No heartbeat from core client for 30 sec - exiting 20:07:40 (3096): No heartbeat from core client for 30 sec - exiting 20:07:41 (3096): No heartbeat from core client for 30 sec - exiting 20:07:42 (3096): No heartbeat from core client for 30 sec - exiting 20:07:43 (3096): No heartbeat from core client for 30 sec - exiting 20:07:44 (3096): No heartbeat from core client for 30 sec - exiting 20:07:45 (3096): No heartbeat from core client for 30 sec - exiting 20:07:46 (3096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4072, iMonCtr=2 Model crash detected, will try to restart... 20:43:11 (5016): No heartbeat from core client for 30 sec - exiting 20:43:12 (5016): No heartbeat from core client for 30 sec - exiting 20:43:13 (5016): No heartbeat from core client for 30 sec - exiting 20:43:14 (5016): No heartbeat from core client for 30 sec - exiting 20:43:15 (5016): No heartbeat from core client for 30 sec - exiting 20:43:16 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:36:41 (1472): No heartbeat from core client for 30 sec - exiting 21:36:42 (1472): No heartbeat from core client for 30 sec - exiting 21:36:43 (1472): No heartbeat from core client for 30 sec - exiting 21:36:44 (1472): No heartbeat from core client for 30 sec - exiting 21:36:45 (1472): No heartbeat from core client for 30 sec - exiting 21:36:46 (1472): No heartbeat from core client for 30 sec - exiting 21:36:47 (1472): No heartbeat from core client for 30 sec - exiting 21:36:48 (1472): No heartbeat from core client for 30 sec - exiting 21:36:49 (1472): No heartbeat from core client for 30 sec - exiting 21:36:50 (1472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5768, iMonCtr=2 Model crash detected, will try to restart... 21:31:35 (5740): No heartbeat from core client for 30 sec - exiting 21:31:36 (5740): No heartbeat from core client for 30 sec - exiting 21:31:37 (5740): No heartbeat from core client for 30 sec - exiting 21:31:38 (5740): No heartbeat from core client for 30 sec - exiting 21:31:39 (5740): No heartbeat from core client for 30 sec - exiting 21:31:40 (5740): No heartbeat from core client for 30 sec - exiting 21:31:41 (5740): No heartbeat from core client for 30 sec - exiting 21:31:42 (5740): No heartbeat from core client for 30 sec - exiting 21:31:43 (5740): No heartbeat from core client for 30 sec - exiting 21:31:44 (5740): No heartbeat from core client for 30 sec - exiting 21:31:45 (5740): No heartbeat from core client for 30 sec - exiting 21:31:46 (5740): No heartbeat from core client for 30 sec - exiting 21:31:47 (5740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3944, selfPID=5136, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_m5og_2012_1_009306886_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Jan 2015 01:01:17	1143523	17593139	hadam3p_anz_m5og_2012_1_009306886_0	23,339	166,973	7.1542
31 Dec 2014 00:06:27	1143523	17593139	hadam3p_anz_m5og_2012_1_009306886_0	11,819	84,933	7.1861