Task 17330732

Name	hadam3p_anz_c0y5_2012_1_009145009_0
Workunit	9275345
Created	31 Oct 2014, 11:15:50 UTC
Sent	31 Oct 2014, 23:00:56 UTC
Report deadline	14 Oct 2015, 4:20:56 UTC
Received	15 Nov 2014, 12:21:17 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1096454
Run time	6 days 1 hours 53 min 16 sec
CPU time	5 days 9 hours 57 min 35 sec
Validate state	Invalid
Credit	3,490.64
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6916, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7916, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4556, iMonCtr=2 21:29:16 (1516): No heartbeat from core client for 30 sec - exiting 21:29:17 (1516): No heartbeat from core client for 30 sec - exiting 21:29:18 (1516): No heartbeat from core client for 30 sec - exiting 21:29:20 (1516): No heartbeat from core client for 30 sec - exiting 21:29:21 (1516): No heartbeat from core client for 30 sec - exiting 21:29:24 (1516): No heartbeat from core client for 30 sec - exiting 21:29:25 (1516): No heartbeat from core client for 30 sec - exiting 21:29:26 (1516): No heartbeat from core client for 30 sec - exiting 21:29:27 (1516): No heartbeat from core client for 30 sec - exiting 21:29:28 (1516): No heartbeat from core client for 30 sec - exiting 21:29:29 (1516): No heartbeat from core client for 30 sec - exiting 21:29:31 (1516): No heartbeat from core client for 30 sec - exiting 21:29:32 (1516): No heartbeat from core client for 30 sec - exiting 21:29:33 (1516): No heartbeat from core client for 30 sec - exiting 21:29:34 (1516): No heartbeat from core client for 30 sec - exiting 21:29:35 (1516): No heartbeat from core client for 30 sec - exiting 21:29:36 (1516): No heartbeat from core client for 30 sec - exiting 21:29:37 (1516): No heartbeat from core client for 30 sec - exiting 21:29:39 (1516): No heartbeat from core client for 30 sec - exiting 21:29:40 (1516): No heartbeat from core client for 30 sec - exiting 21:29:41 (1516): No heartbeat from core client for 30 sec - exiting 21:29:42 (1516): No heartbeat from core client for 30 sec - exiting 21:29:43 (1516): No heartbeat from core client for 30 sec - exiting 21:29:44 (1516): No heartbeat from core client for 30 sec - exiting 21:29:45 (1516): No heartbeat from core client for 30 sec - exiting 21:29:46 (1516): No heartbeat from core client for 30 sec - exiting 21:29:47 (1516): No heartbeat from core client for 30 sec - exiting 21:29:48 (1516): No heartbeat from core client for 30 sec - exiting 21:29:49 (1516): No heartbeat from core client for 30 sec - exiting 21:29:50 (1516): No heartbeat from core client for 30 sec - exiting 21:29:51 (1516): No heartbeat from core client for 30 sec - exiting 21:29:52 (1516): No heartbeat from core client for 30 sec - exiting 21:29:53 (1516): No heartbeat from core client for 30 sec - exiting 21:29:54 (1516): No heartbeat from core client for 30 sec - exiting 21:29:55 (1516): No heartbeat from core client for 30 sec - exiting 21:29:56 (1516): No heartbeat from core client for 30 sec - exiting 21:29:57 (1516): No heartbeat from core client for 30 sec - exiting 21:29:58 (1516): No heartbeat from core client for 30 sec - exiting 21:29:59 (1516): No heartbeat from core client for 30 sec - exiting 21:30:00 (1516): No heartbeat from core client for 30 sec - exiting 21:30:01 (1516): No heartbeat from core client for 30 sec - exiting 21:30:02 (1516): No heartbeat from core client for 30 sec - exiting 21:30:03 (1516): No heartbeat from core client for 30 sec - exiting 21:30:04 (1516): No heartbeat from core client for 30 sec - exiting 21:30:05 (1516): No heartbeat from core client for 30 sec - exiting 21:30:06 (1516): No heartbeat from core client for 30 sec - exiting 21:30:07 (1516): No heartbeat from core client for 30 sec - exiting 21:30:09 (1516): No heartbeat from core client for 30 sec - exiting 21:30:10 (1516): No heartbeat from core client for 30 sec - exiting 21:30:11 (1516): No heartbeat from core client for 30 sec - exiting 21:30:12 (1516): No heartbeat from core client for 30 sec - exiting 21:30:13 (1516): No heartbeat from core client for 30 sec - exiting 21:30:14 (1516): No heartbeat from core client for 30 sec - exiting 21:30:15 (1516): No heartbeat from core client for 30 sec - exiting 21:30:16 (1516): No heartbeat from core client for 30 sec - exiting 21:30:17 (1516): No heartbeat from core client for 30 sec - exiting 21:30:18 (1516): No heartbeat from core client for 30 sec - exiting 21:30:19 (1516): No heartbeat from core client for 30 sec - exiting 21:30:20 (1516): No heartbeat from core client for 30 sec - exiting 21:30:21 (1516): No heartbeat from core client for 30 sec - exiting 21:30:22 (1516): No heartbeat from core client for 30 sec - exiting 21:30:23 (1516): No heartbeat from core client for 30 sec - exiting 21:30:24 (1516): No heartbeat from core client for 30 sec - exiting 21:30:25 (1516): No heartbeat from core client for 30 sec - exiting 21:30:26 (1516): No heartbeat from core client for 30 sec - exiting 21:30:27 (1516): No heartbeat from core client for 30 sec - exiting 21:30:28 (1516): No heartbeat from core client for 30 sec - exiting 21:30:29 (1516): No heartbeat from core client for 30 sec - exiting 21:30:30 (1516): No heartbeat from core client for 30 sec - exiting 21:30:31 (1516): No heartbeat from core client for 30 sec - exiting 21:30:32 (1516): No heartbeat from core client for 30 sec - exiting 21:30:33 (1516): No heartbeat from core client for 30 sec - exiting 21:30:34 (1516): No heartbeat from core client for 30 sec - exiting 21:30:35 (1516): No heartbeat from core client for 30 sec - exiting 21:30:36 (1516): No heartbeat from core client for 30 sec - exiting 21:30:37 (1516): No heartbeat from core client for 30 sec - exiting 21:30:38 (1516): No heartbeat from core client for 30 sec - exiting 21:30:39 (1516): No heartbeat from core client for 30 sec - exiting 21:30:40 (1516): No heartbeat from core client for 30 sec - exiting 21:30:41 (1516): No heartbeat from core client for 30 sec - exiting 21:30:42 (1516): No heartbeat from core client for 30 sec - exiting 21:30:43 (1516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:54:42 (4620): No heartbeat from core client for 30 sec - exiting 19:54:43 (4620): No heartbeat from core client for 30 sec - exiting 19:54:44 (4620): No heartbeat from core client for 30 sec - exiting 19:54:45 (4620): No heartbeat from core client for 30 sec - exiting 19:54:46 (4620): No heartbeat from core client for 30 sec - exiting 19:54:47 (4620): No heartbeat from core client for 30 sec - exiting 19:54:48 (4620): No heartbeat from core client for 30 sec - exiting 19:54:49 (4620): No heartbeat from core client for 30 sec - exiting 19:54:50 (4620): No heartbeat from core client for 30 sec - exiting 19:54:51 (4620): No heartbeat from core client for 30 sec - exiting 19:54:53 (4620): No heartbeat from core client for 30 sec - exiting 19:54:54 (4620): No heartbeat from core client for 30 sec - exiting 19:54:55 (4620): No heartbeat from core client for 30 sec - exiting 19:54:56 (4620): No heartbeat from core client for 30 sec - exiting 19:54:57 (4620): No heartbeat from core client for 30 sec - exiting 19:54:58 (4620): No heartbeat from core client for 30 sec - exiting 19:54:59 (4620): No heartbeat from core client for 30 sec - exiting 19:55:00 (4620): No heartbeat from core client for 30 sec - exiting 19:55:01 (4620): No heartbeat from core client for 30 sec - exiting 19:55:02 (4620): No heartbeat from core client for 30 sec - exiting 19:55:03 (4620): No heartbeat from core client for 30 sec - exiting 19:55:05 (4620): No heartbeat from core client for 30 sec - exiting 19:55:06 (4620): No heartbeat from core client for 30 sec - exiting 19:55:07 (4620): No heartbeat from core client for 30 sec - exiting 19:55:08 (4620): No heartbeat from core client for 30 sec - exiting 19:55:09 (4620): No heartbeat from core client for 30 sec - exiting 19:55:10 (4620): No heartbeat from core client for 30 sec - exiting 19:55:11 (4620): No heartbeat from core client for 30 sec - exiting 19:55:12 (4620): No heartbeat from core client for 30 sec - exiting 19:55:13 (4620): No heartbeat from core client for 30 sec - exiting 19:55:14 (4620): No heartbeat from core client for 30 sec - exiting 19:55:15 (4620): No heartbeat from core client for 30 sec - exiting 19:55:17 (4620): No heartbeat from core client for 30 sec - exiting 19:55:18 (4620): No heartbeat from core client for 30 sec - exiting 19:55:19 (4620): No heartbeat from core client for 30 sec - exiting 19:55:20 (4620): No heartbeat from core client for 30 sec - exiting 19:55:21 (4620): No heartbeat from core client for 30 sec - exiting 19:55:22 (4620): No heartbeat from core client for 30 sec - exiting 19:55:23 (4620): No heartbeat from core client for 30 sec - exiting 19:55:24 (4620): No heartbeat from core client for 30 sec - exiting 19:55:25 (4620): No heartbeat from core client for 30 sec - exiting 19:55:26 (4620): No heartbeat from core client for 30 sec - exiting 19:55:27 (4620): No heartbeat from core client for 30 sec - exiting 19:55:29 (4620): No heartbeat from core client for 30 sec - exiting 19:55:30 (4620): No heartbeat from core client for 30 sec - exiting 19:55:31 (4620): No heartbeat from core client for 30 sec - exiting 19:55:32 (4620): No heartbeat from core client for 30 sec - exiting 19:55:33 (4620): No heartbeat from core client for 30 sec - exiting 19:55:34 (4620): No heartbeat from core client for 30 sec - exiting 19:55:35 (4620): No heartbeat from core client for 30 sec - exiting 19:55:36 (4620): No heartbeat from core client for 30 sec - exiting 19:55:37 (4620): No heartbeat from core client for 30 sec - exiting 19:55:38 (4620): No heartbeat from core client for 30 sec - exiting 19:55:40 (4620): No heartbeat from core client for 30 sec - exiting 19:55:41 (4620): No heartbeat from core client for 30 sec - exiting 19:55:42 (4620): No heartbeat from core client for 30 sec - exiting 19:55:43 (4620): No heartbeat from core client for 30 sec - exiting 19:55:44 (4620): No heartbeat from core client for 30 sec - exiting 19:55:45 (4620): No heartbeat from core client for 30 sec - exiting 19:55:46 (4620): No heartbeat from core client for 30 sec - exiting 19:55:47 (4620): No heartbeat from core client for 30 sec - exiting 19:55:48 (4620): No heartbeat from core client for 30 sec - exiting 19:55:49 (4620): No heartbeat from core client for 30 sec - exiting 19:55:51 (4620): No heartbeat from core client for 30 sec - exiting 19:55:52 (4620): No heartbeat from core client for 30 sec - exiting 19:55:53 (4620): No heartbeat from core client for 30 sec - exiting 19:55:54 (4620): No heartbeat from core client for 30 sec - exiting 19:55:55 (4620): No heartbeat from core client for 30 sec - exiting 19:55:56 (4620): No heartbeat from core client for 30 sec - exiting 19:55:57 (4620): No heartbeat from core client for 30 sec - exiting 19:55:58 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:52:16 (5072): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 22:52:18 (5072): No heartbeat from core client for 30 sec - exiting 22:52:19 (5072): No heartbeat from core client for 30 sec - exiting 22:52:20 (5072): No heartbeat from core client for 30 sec - exiting BUFFIN: C I/O Error ferror - Unit 63 - Return code = 32 Model crashed: </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_c0y5_2012_1_009145009_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c0y5_2012_1_009145009_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c0y5_2012_1_009145009_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c0y5_2012_1_009145009_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_c0y5_2012_1_009145009_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Nov 2014 15:22:03	1096454	17330732	hadam3p_anz_c0y5_2012_1_009145009_0	80,939	425,610	5.2584
12 Nov 2014 20:14:18	1096454	17330732	hadam3p_anz_c0y5_2012_1_009145009_0	69,419	363,576	5.2374
11 Nov 2014 05:31:04	1096454	17330732	hadam3p_anz_c0y5_2012_1_009145009_0	57,899	303,221	5.2371
09 Nov 2014 12:03:16	1096454	17330732	hadam3p_anz_c0y5_2012_1_009145009_0	46,379	244,563	5.2731
07 Nov 2014 14:30:18	1096454	17330732	hadam3p_anz_c0y5_2012_1_009145009_0	34,859	184,824	5.3020
06 Nov 2014 12:52:55	1096454	17330732	hadam3p_anz_c0y5_2012_1_009145009_0	23,339	128,258	5.4954
05 Nov 2014 00:53:21	1096454	17330732	hadam3p_anz_c0y5_2012_1_009145009_0	11,819	67,864	5.7419