Task 18901417

Name	hadam3p_anz_w02u_1987_1_010173778_0
Workunit	10127934
Created	12 Sep 2015, 19:35:29 UTC
Sent	13 Sep 2015, 2:14:56 UTC
Report deadline	25 Aug 2016, 7:34:56 UTC
Received	26 Sep 2015, 16:06:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1360541
Run time	7 days 1 hours 7 min 22 sec
CPU time	5 days 16 hours 26 min 38 sec
Validate state	Invalid
Credit	2,989.68
Device peak FLOPS	2.27 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.9</core_client_version> <![CDATA[ <stderr_txt> 06:28:58 (8544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:28:59 (8544): No heartbeat from core client for 30 sec - exiting 06:29:00 (8544): No heartbeat from core client for 30 sec - exiting 06:29:01 (8544): No heartbeat from core client for 30 sec - exiting 06:29:02 (8544): No heartbeat from core client for 30 sec - exiting 10:50:15 (9908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:50:16 (9908): No heartbeat from core client for 30 sec - exiting 10:50:17 (9908): No heartbeat from core client for 30 sec - exiting 10:50:18 (9908): No heartbeat from core client for 30 sec - exiting 10:50:19 (9908): No heartbeat from core client for 30 sec - exiting 10:50:20 (9908): No heartbeat from core client for 30 sec - exiting 10:50:21 (9908): No heartbeat from core client for 30 sec - exiting 10:50:22 (9908): No heartbeat from core client for 30 sec - exiting 10:50:23 (9908): No heartbeat from core client for 30 sec - exiting 10:50:24 (9908): No heartbeat from core client for 30 sec - exiting 10:50:25 (9908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 13:52:57 (5920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:52:58 (5920): No heartbeat from core client for 30 sec - exiting 13:52:59 (5920): No heartbeat from core client for 30 sec - exiting 13:53:00 (5920): No heartbeat from core client for 30 sec - exiting 13:53:01 (5920): No heartbeat from core client for 30 sec - exiting 13:53:02 (5920): No heartbeat from core client for 30 sec - exiting 13:53:03 (5920): No heartbeat from core client for 30 sec - exiting 13:53:04 (5920): No heartbeat from core client for 30 sec - exiting 13:53:05 (5920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 16:58:32 (11308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:24:18 (8544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:34 (8396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:27 (2188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:28 (2188): No heartbeat from core client for 30 sec - exiting 05:20:29 (2188): No heartbeat from core client for 30 sec - exiting 05:20:30 (2188): No heartbeat from core client for 30 sec - exiting 05:20:31 (2188): No heartbeat from core client for 30 sec - exiting 05:20:33 (2188): No heartbeat from core client for 30 sec - exiting 05:20:34 (2188): No heartbeat from core client for 30 sec - exiting 05:20:35 (2188): No heartbeat from core client for 30 sec - exiting 05:20:36 (2188): No heartbeat from core client for 30 sec - exiting 05:20:37 (2188): No heartbeat from core client for 30 sec - exiting 05:20:38 (2188): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9968, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7188, iMonCtr=2 Model crash detected, will try to restart... 22:08:33 (4100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:41:24 (4948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:01:19 (4480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:30:11 (5656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:15:21 (8156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:22 (8156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 21:31:26 (2136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:31:27 (2136): No heartbeat from core client for 30 sec - exiting 21:31:28 (2136): No heartbeat from core client for 30 sec - exiting 21:31:29 (2136): No heartbeat from core client for 30 sec - exiting 21:31:30 (2136): No heartbeat from core client for 30 sec - exiting 21:31:31 (2136): No heartbeat from core client for 30 sec - exiting 21:31:32 (2136): No heartbeat from core client for 30 sec - exiting 21:31:33 (2136): No heartbeat from core client for 30 sec - exiting 21:31:34 (2136): No heartbeat from core client for 30 sec - exiting 21:31:35 (2136): No heartbeat from core client for 30 sec - exiting 21:31:36 (2136): No heartbeat from core client for 30 sec - exiting 01:04:58 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:06:10 (7600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:06:11 (7600): No heartbeat from core client for 30 sec - exiting 03:06:12 (7600): No heartbeat from core client for 30 sec - exiting 03:06:13 (7600): No heartbeat from core client for 30 sec - exiting 03:06:14 (7600): No heartbeat from core client for 30 sec - exiting 03:06:15 (7600): No heartbeat from core client for 30 sec - exiting 03:06:16 (7600): No heartbeat from core client for 30 sec - exiting 03:06:17 (7600): No heartbeat from core client for 30 sec - exiting 03:06:18 (7600): No heartbeat from core client for 30 sec - exiting 03:06:19 (7600): No heartbeat from core client for 30 sec - exiting 03:06:20 (7600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 21:10:50 (7400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:10:51 (7400): No heartbeat from core client for 30 sec - exiting 21:10:52 (7400): No heartbeat from core client for 30 sec - exiting 21:10:53 (7400): No heartbeat from core client for 30 sec - exiting 21:10:54 (7400): No heartbeat from core client for 30 sec - exiting 21:10:55 (7400): No heartbeat from core client for 30 sec - exiting 21:10:56 (7400): No heartbeat from core client for 30 sec - exiting 21:10:57 (7400): No heartbeat from core client for 30 sec - exiting 21:10:58 (7400): No heartbeat from core client for 30 sec - exiting 21:10:59 (7400): No heartbeat from core client for 30 sec - exiting 21:11:00 (7400): No heartbeat from core client for 30 sec - exiting 01:15:59 (1296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:16:00 (1296): No heartbeat from core client for 30 sec - exiting 01:16:01 (1296): No heartbeat from core client for 30 sec - exiting 01:16:02 (1296): No heartbeat from core client for 30 sec - exiting 01:16:03 (1296): No heartbeat from core client for 30 sec - exiting 01:16:04 (1296): No heartbeat from core client for 30 sec - exiting 01:16:05 (1296): No heartbeat from core client for 30 sec - exiting 01:16:06 (1296): No heartbeat from core client for 30 sec - exiting 01:16:07 (1296): No heartbeat from core client for 30 sec - exiting 01:16:08 (1296): No heartbeat from core client for 30 sec - exiting 01:16:09 (1296): No heartbeat from core client for 30 sec - exiting 13:13:55 (9756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:13:56 (9756): No heartbeat from core client for 30 sec - exiting 13:13:57 (9756): No heartbeat from core client for 30 sec - exiting 13:13:58 (9756): No heartbeat from core client for 30 sec - exiting 13:13:59 (9756): No heartbeat from core client for 30 sec - exiting 13:14:00 (9756): No heartbeat from core client for 30 sec - exiting 13:14:01 (9756): No heartbeat from core client for 30 sec - exiting 13:14:02 (9756): No heartbeat from core client for 30 sec - exiting 13:14:03 (9756): No heartbeat from core client for 30 sec - exiting 13:14:04 (9756): No heartbeat from core client for 30 sec - exiting 13:14:05 (9756): No heartbeat from core client for 30 sec - exiting 18:21:19 (11372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:15:22 (10488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:58:35 (2532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:37 (12728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:38 (12728): No heartbeat from core client for 30 sec - exiting 04:49:39 (12728): No heartbeat from core client for 30 sec - exiting 04:49:40 (12728): No heartbeat from core client for 30 sec - exiting 04:49:41 (12728): No heartbeat from core client for 30 sec - exiting 04:49:42 (12728): No heartbeat from core client for 30 sec - exiting 04:49:43 (12728): No heartbeat from core client for 30 sec - exiting 04:49:44 (12728): No heartbeat from core client for 30 sec - exiting 04:49:45 (12728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:42:34 (6044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:25:57 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:44:26 (7316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:17:08 (5976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3752, selfPID=3752, iMonCtr=2 03:16:17 (7392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:46:40 (5508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:24:26 (2072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5912, iMonCtr=2 21:21:12 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:18:14 (6448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:38:33 (2696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:34 (2696): No heartbeat from core client for 30 sec - exiting 17:38:35 (2696): No heartbeat from core client for 30 sec - exiting 17:38:36 (2696): No heartbeat from core client for 30 sec - exiting 17:38:37 (2696): No heartbeat from core client for 30 sec - exiting 17:38:38 (2696): No heartbeat from core client for 30 sec - exiting 17:38:39 (2696): No heartbeat from core client for 30 sec - exiting 17:38:40 (2696): No heartbeat from core client for 30 sec - exiting 17:38:41 (2696): No heartbeat from core client for 30 sec - exiting 18:18:26 (6752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:38:11 (1376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:05:58 (4348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:05:59 (4348): No heartbeat from core client for 30 sec - exiting 02:06:00 (4348): No heartbeat from core client for 30 sec - exiting 02:06:01 (4348): No heartbeat from core client for 30 sec - exiting 02:06:02 (4348): No heartbeat from core client for 30 sec - exiting 02:06:03 (4348): No heartbeat from core client for 30 sec - exiting 02:06:04 (4348): No heartbeat from core client for 30 sec - exiting 02:06:05 (4348): No heartbeat from core client for 30 sec - exiting 02:06:06 (4348): No heartbeat from core client for 30 sec - exiting 02:06:07 (4348): No heartbeat from core client for 30 sec - exiting 02:06:08 (4348): No heartbeat from core client for 30 sec - exiting 04:54:30 (7644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:54:31 (7644): No heartbeat from core client for 30 sec - exiting 04:54:32 (7644): No heartbeat from core client for 30 sec - exiting 04:54:33 (7644): No heartbeat from core client for 30 sec - exiting 17:05:06 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8920, selfPID=8920, iMonCtr=2 17:05:07 (4024): No heartbeat from core client for 30 sec - exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8912, selfPID=8912, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8912, selfPID=8828, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_w02u_1987_1_010173778_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02u_1987_1_010173778_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02u_1987_1_010173778_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02u_1987_1_010173778_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02u_1987_1_010173778_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_w02u_1987_1_010173778_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Sep 2015 17:53:41	1360541	18901417	hadam3p_anz_w02u_1987_1_010173778_0	69,323	432,447	6.2381
23 Sep 2015 21:57:41	1360541	18901417	hadam3p_anz_w02u_1987_1_010173778_0	57,803	362,282	6.2675
22 Sep 2015 02:12:13	1360541	18901417	hadam3p_anz_w02u_1987_1_010173778_0	46,283	290,883	6.2849
19 Sep 2015 11:16:47	1360541	18901417	hadam3p_anz_w02u_1987_1_010173778_0	34,859	220,104	6.3141
17 Sep 2015 18:23:06	1360541	18901417	hadam3p_anz_w02u_1987_1_010173778_0	23,339	147,263	6.3097
15 Sep 2015 18:29:39	1360541	18901417	hadam3p_anz_w02u_1987_1_010173778_0	11,819	75,210	6.3635