Task 19228372

Name	hadam3p_anz_a2gf_201412_12_313_010278384_0
Workunit	10278384
Created	28 Jan 2016, 10:51:41 UTC
Sent	28 Jan 2016, 12:12:05 UTC
Report deadline	9 Jan 2017, 17:32:05 UTC
Received	12 Feb 2016, 5:44:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1362789
Run time	2 days 2 hours 13 min 26 sec
CPU time	2 days 2 hours 9 min 31 sec
Validate state	Invalid
Credit	2,497.00
Device peak FLOPS	4.33 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.9</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:16:38 (33972): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:33:25 (23452): start_timer_thread(): CreateThread() failed, errno 0 20:33:27 (31408): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:08:05 (24668): start_timer_thread(): CreateThread() failed, errno 0 23:08:06 (24724): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:32:01 (30604): start_timer_thread(): CreateThread() failed, errno 0 23:32:02 (34284): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 01:48:49 (40252): start_timer_thread(): CreateThread() failed, errno 0 01:48:51 (32156): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 02:16:01 (25588): start_timer_thread(): CreateThread() failed, errno 0 02:16:02 (34552): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 02:22:14 (30624): start_timer_thread(): CreateThread() failed, errno 0 02:22:16 (14504): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 04:27:37 (33952): start_timer_thread(): CreateThread() failed, errno 0 04:27:39 (23316): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:51:42 (26952): start_timer_thread(): CreateThread() failed, errno 0 06:51:44 (27544): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 07:46:02 (36020): start_timer_thread(): CreateThread() failed, errno 0 07:46:03 (31820): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 07:49:46 (14524): start_timer_thread(): CreateThread() failed, errno 0 07:49:48 (39012): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 10:09:35 (32412): start_timer_thread(): CreateThread() failed, errno 0 10:09:36 (24748): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:08:19 (41788): start_timer_thread(): CreateThread() failed, errno 0 13:08:21 (4164): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:11:49 (40224): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:43:32 (1448): start_timer_thread(): CreateThread() failed, errno 0 15:43:34 (30744): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:07:25 (34640): start_timer_thread(): CreateThread() failed, errno 0 22:07:27 (10564): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 03:29:28 (34532): start_timer_thread(): CreateThread() failed, errno 0 03:29:29 (2988): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 11:04:27 (42224): start_timer_thread(): CreateThread() failed, errno 0 11:04:28 (22952): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:25:00 (42572): start_timer_thread(): CreateThread() failed, errno 0 14:25:02 (16008): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:27:26 (24460): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:52:51 (14268): start_timer_thread(): CreateThread() failed, errno 0 18:52:52 (16508): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 02:33:58 (42672): start_timer_thread(): CreateThread() failed, errno 0 02:33:59 (25704): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:14:58 (24272): start_timer_thread(): CreateThread() failed, errno 0 12:15:00 (632): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:24:58 (38584): start_timer_thread(): CreateThread() failed, errno 0 17:25:00 (36028): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:00:42 (38784): start_timer_thread(): CreateThread() failed, errno 0 23:00:44 (16328): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:33:41 (16080): start_timer_thread(): CreateThread() failed, errno 0 06:33:42 (16280): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:08:59 (38744): start_timer_thread(): CreateThread() failed, errno 0 13:09:01 (17412): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 13:27:47 (39332): start_timer_thread(): CreateThread() failed, errno 0 13:27:49 (15200): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:09:45 (38968): start_timer_thread(): CreateThread() failed, errno 0 14:09:46 (39356): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:41:10 (11620): start_timer_thread(): CreateThread() failed, errno 0 20:41:11 (25220): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:07:22 (15480): start_timer_thread(): CreateThread() failed, errno 0 21:07:23 (23996): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:05:14 (31160): start_timer_thread(): CreateThread() failed, errno 0 21:05:16 (5244): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:23:05 (7560): start_timer_thread(): CreateThread() failed, errno 0 21:23:06 (22656): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:09:39 (11668): start_timer_thread(): CreateThread() failed, errno 0 18:09:40 (9172): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:15:00 (12920): start_timer_thread(): CreateThread() failed, errno 0 19:15:02 (8560): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:20:22 (10344): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:32:09 (36940): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:45:11 (27884): start_timer_thread(): CreateThread() failed, errno 0 17:45:12 (15200): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:49:13 (7644): start_timer_thread(): CreateThread() failed, errno 0 17:49:15 (16288): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:56:00 (25404): start_timer_thread(): CreateThread() failed, errno 0 17:56:01 (29124): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:12:18 (26212): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:41:21 (20920): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:45:55 (28512): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:19:57 (34824): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:19:20 (5404): start_timer_thread(): CreateThread() failed, errno 0 21:19:22 (23564): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:28:02 (1496): start_timer_thread(): CreateThread() failed, errno 0 22:28:03 (27920): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:50:27 (46708): start_timer_thread(): CreateThread() failed, errno 0 17:50:28 (46560): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:18:11 (47936): start_timer_thread(): CreateThread() failed, errno 0 18:18:13 (47552): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:55:57 (44736): start_timer_thread(): CreateThread() failed, errno 0 18:55:59 (46824): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:13:30 (45772): start_timer_thread(): CreateThread() failed, errno 0 19:13:32 (47980): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:26:33 (47492): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:30:51 (35640): start_timer_thread(): CreateThread() failed, errno 0 19:30:53 (43664): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:41:40 (30388): start_timer_thread(): CreateThread() failed, errno 0 19:41:41 (47160): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:40:55 (48080): No heartbeat from core client for 30 sec - exiting 20:40:56 (48080): No heartbeat from core client for 30 sec - exiting 20:40:57 (48080): No heartbeat from core client for 30 sec - exiting 20:40:58 (48080): No heartbeat from core client for 30 sec - exiting 20:40:59 (48080): No heartbeat from core client for 30 sec - exiting 20:41:00 (48080): No heartbeat from core client for 30 sec - exiting 20:41:01 (48080): No heartbeat from core client for 30 sec - exiting 20:41:02 (48080): No heartbeat from core client for 30 sec - exiting 20:41:03 (48080): No heartbeat from core client for 30 sec - exiting 20:41:04 (48080): No heartbeat from core client for 30 sec - exiting 20:41:05 (48080): No heartbeat from core client for 30 sec - exiting 20:41:06 (48080): No heartbeat from core client for 30 sec - exiting 20:41:07 (48080): No heartbeat from core client for 30 sec - exiting 20:41:08 (48080): No heartbeat from core client for 30 sec - exiting 20:41:09 (48080): No heartbeat from core client for 30 sec - exiting 20:41:10 (48080): No heartbeat from core client for 30 sec - exiting 20:41:11 (48080): No heartbeat from core client for 30 sec - exiting 20:41:12 (48080): No heartbeat from core client for 30 sec - exiting 20:41:13 (48080): No heartbeat from core client for 30 sec - exiting 20:41:14 (48080): No heartbeat from core client for 30 sec - exiting 20:41:15 (48080): No heartbeat from core client for 30 sec - exiting 20:41:16 (48080): No heartbeat from core client for 30 sec - exiting 20:41:17 (48080): No heartbeat from core client for 30 sec - exiting 20:41:18 (48080): No heartbeat from core client for 30 sec - exiting 20:41:19 (48080): No heartbeat from core client for 30 sec - exiting 20:41:20 (48080): No heartbeat from core client for 30 sec - exiting 20:41:21 (48080): No heartbeat from core client for 30 sec - exiting 20:41:22 (48080): No heartbeat from core client for 30 sec - exiting 20:41:23 (48080): No heartbeat from core client for 30 sec - exiting 20:41:24 (48080): No heartbeat from core client for 30 sec - exiting 20:41:25 (48080): No heartbeat from core client for 30 sec - exiting 20:41:26 (48080): No heartbeat from core client for 30 sec - exiting 20:41:27 (48080): No heartbeat from core client for 30 sec - exiting 20:41:28 (48080): No heartbeat from core client for 30 sec - exiting 20:41:29 (48080): No heartbeat from core client for 30 sec - exiting 20:41:30 (48080): No heartbeat from core client for 30 sec - exiting 20:41:31 (48080): No heartbeat from core client for 30 sec - exiting 20:41:32 (48080): No heartbeat from core client for 30 sec - exiting 20:41:33 (48080): No heartbeat from core client for 30 sec - exiting 20:41:34 (48080): No heartbeat from core client for 30 sec - exiting 20:41:35 (48080): No heartbeat from core client for 30 sec - exiting 20:41:36 (48080): No heartbeat from core client for 30 sec - exiting 20:41:37 (48080): No heartbeat from core client for 30 sec - exiting 20:41:38 (48080): No heartbeat from core client for 30 sec - exiting 20:41:39 (48080): No heartbeat from core client for 30 sec - exiting 20:41:40 (48080): No heartbeat from core client for 30 sec - exiting 20:41:41 (48080): No heartbeat from core client for 30 sec - exiting 20:41:42 (48080): No heartbeat from core client for 30 sec - exiting 20:41:43 (48080): No heartbeat from core client for 30 sec - exiting 20:41:44 (48080): No heartbeat from core client for 30 sec - exiting 20:41:45 (48080): No heartbeat from core client for 30 sec - exiting 20:41:46 (48080): No heartbeat from core client for 30 sec - exiting 20:41:47 (48080): No heartbeat from core client for 30 sec - exiting 20:41:48 (48080): No heartbeat from core client for 30 sec - exiting 20:41:49 (48080): No heartbeat from core client for 30 sec - exiting 20:41:50 (48080): No heartbeat from core client for 30 sec - exiting 20:41:51 (48080): No heartbeat from core client for 30 sec - exiting 20:41:52 (48080): No heartbeat from core client for 30 sec - exiting 20:41:53 (48080): No heartbeat from core client for 30 sec - exiting 20:41:54 (48080): No heartbeat from core client for 30 sec - exiting 20:41:55 (48080): No heartbeat from core client for 30 sec - exiting 20:41:56 (48080): No heartbeat from core client for 30 sec - exiting 20:41:57 (48080): No heartbeat from core client for 30 sec - exiting 20:41:58 (48080): No heartbeat from core client for 30 sec - exiting 20:41:59 (48080): No heartbeat from core client for 30 sec - exiting 20:42:00 (48080): No heartbeat from core client for 30 sec - exiting 20:42:01 (48080): No heartbeat from core client for 30 sec - exiting 20:42:02 (48080): No heartbeat from core client for 30 sec - exiting 20:42:04 (48080): No heartbeat from core client for 30 sec - exiting 20:42:05 (48080): No heartbeat from core client for 30 sec - exiting 20:42:06 (48080): No heartbeat from core client for 30 sec - exiting 20:42:07 (48080): No heartbeat from core client for 30 sec - exiting 20:42:08 (48080): No heartbeat from core client for 30 sec - exiting 20:42:09 (48080): No heartbeat from core client for 30 sec - exiting 20:42:10 (48080): No heartbeat from core client for 30 sec - exiting 20:42:11 (48080): No heartbeat from core client for 30 sec - exiting 20:42:12 (48080): No heartbeat from core client for 30 sec - exiting 20:42:13 (48080): No heartbeat from core client for 30 sec - exiting 20:42:14 (48080): No heartbeat from core client for 30 sec - exiting 20:42:15 (48080): No heartbeat from core client for 30 sec - exiting 20:42:16 (48080): No heartbeat from core client for 30 sec - exiting 20:42:17 (48080): No heartbeat from core client for 30 sec - exiting 20:42:18 (48080): No heartbeat from core client for 30 sec - exiting 20:42:19 (48080): No heartbeat from core client for 30 sec - exiting 20:42:20 (48080): No heartbeat from core client for 30 sec - exiting 20:42:21 (48080): No heartbeat from core client for 30 sec - exiting 20:42:22 (48080): No heartbeat from core client for 30 sec - exiting 20:42:23 (48080): No heartbeat from core client for 30 sec - exiting 20:42:24 (48080): No heartbeat from core client for 30 sec - exiting 20:42:25 (48080): No heartbeat from core client for 30 sec - exiting 20:42:26 (48080): No heartbeat from core client for 30 sec - exiting 20:42:27 (48080): No heartbeat from core client for 30 sec - exiting 20:42:28 (48080): No heartbeat from core client for 30 sec - exiting 20:42:29 (48080): No heartbeat from core client for 30 sec - exiting 20:42:30 (48080): No heartbeat from core client for 30 sec - exiting 20:42:31 (48080): No heartbeat from core client for 30 sec - exiting 20:42:32 (48080): No heartbeat from core client for 30 sec - exiting 20:42:33 (48080): No heartbeat from core client for 30 sec - exiting 20:42:34 (48080): No heartbeat from core client for 30 sec - exiting 20:42:35 (48080): No heartbeat from core client for 30 sec - exiting 20:42:36 (48080): No heartbeat from core client for 30 sec - exiting 20:42:37 (48080): No heartbeat from core client for 30 sec - exiting 20:42:38 (48080): No heartbeat from core client for 30 sec - exiting 20:42:39 (48080): No heartbeat from core client for 30 sec - exiting 20:42:40 (48080): No heartbeat from core client for 30 sec - exiting 20:42:41 (48080): No heartbeat from core client for 30 sec - exiting 20:42:42 (48080): No heartbeat from core client for 30 sec - exiting 20:42:43 (48080): No heartbeat from core client for 30 sec - exiting 20:42:44 (48080): No heartbeat from core client for 30 sec - exiting 20:42:45 (48080): No heartbeat from core client for 30 sec - exiting 20:42:46 (48080): No heartbeat from core client for 30 sec - exiting 20:42:47 (48080): No heartbeat from core client for 30 sec - exiting 20:42:48 (48080): No heartbeat from core client for 30 sec - exiting 20:42:49 (48080): No heartbeat from core client for 30 sec - exiting 20:42:50 (48080): No heartbeat from core client for 30 sec - exiting 20:42:51 (48080): No heartbeat from core client for 30 sec - exiting 20:42:52 (48080): No heartbeat from core client for 30 sec - exiting 20:42:53 (48080): No heartbeat from core client for 30 sec - exiting 20:42:54 (48080): No heartbeat from core client for 30 sec - exiting 20:42:55 (48080): No heartbeat from core client for 30 sec - exiting 20:42:56 (48080): No heartbeat from core client for 30 sec - exiting 20:42:57 (48080): No heartbeat from core client for 30 sec - exiting 20:42:58 (48080): No heartbeat from core client for 30 sec - exiting 20:42:59 (48080): No heartbeat from core client for 30 sec - exiting 20:43:00 (48080): No heartbeat from core client for 30 sec - exiting 20:43:01 (48080): No heartbeat from core client for 30 sec - exiting 20:43:02 (48080): No heartbeat from core client for 30 sec - exiting 20:43:02 (47200): start_timer_thread(): CreateThread() failed, errno 0 20:43:02 (46732): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - No 'heartbeat' from BOINC... 20:44:49 (45748): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=45748, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=45952, selfPID=45952, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=45952, selfPID=46792, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_a2gf_201412_12_313_010278384_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2gf_201412_12_313_010278384_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2gf_201412_12_313_010278384_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2gf_201412_12_313_010278384_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2gf_201412_12_313_010278384_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2gf_201412_12_313_010278384_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a2gf_201412_12_313_010278384_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Feb 2016 18:24:57	1362789	19228372	hadam3p_anz_a2gf_201412_12_313_010278384_0	57,899	152,365	2.6316
11 Feb 2016 07:09:02	1362789	19228372	hadam3p_anz_a2gf_201412_12_313_010278384_0	46,379	112,235	2.4200
09 Feb 2016 09:11:46	1362789	19228372	hadam3p_anz_a2gf_201412_12_313_010278384_0	34,859	85,316	2.4475
06 Feb 2016 12:53:37	1362789	19228372	hadam3p_anz_a2gf_201412_12_313_010278384_0	23,339	62,249	2.6672
02 Feb 2016 21:04:01	1362789	19228372	hadam3p_anz_a2gf_201412_12_313_010278384_0	11,819	31,542	2.6688