Task 16459025

Name	hadam3p_anz_o8dh_2012_1_008627635_0
Workunit	8774147
Created	2 Apr 2014, 19:41:26 UTC
Sent	14 Apr 2014, 9:20:16 UTC
Report deadline	27 Mar 2015, 14:40:16 UTC
Received	29 Apr 2014, 15:10:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1322841
Run time	4 days 9 hours 30 min 29 sec
CPU time	3 days 17 hours 4 min 35 sec
Validate state	Invalid
Credit	1,503.36
Device peak FLOPS	2.75 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.3.11</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7628, selfPID=4992, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9752, selfPID=3264, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish 09:55:44 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8756, selfPID=6492, iMonCtr=1 Model crash detected, will try to restart... 13:26:09 (4824): No heartbeat from core client for 30 sec - exiting 13:26:10 (4824): No heartbeat from core client for 30 sec - exiting 13:26:11 (4824): No heartbeat from core client for 30 sec - exiting 13:26:12 (4824): No heartbeat from core client for 30 sec - exiting 13:26:13 (4824): No heartbeat from core client for 30 sec - exiting 13:26:14 (4824): No heartbeat from core client for 30 sec - exiting 13:26:15 (4824): No heartbeat from core client for 30 sec - exiting 13:26:17 (4824): No heartbeat from core client for 30 sec - exiting 13:26:18 (4824): No heartbeat from core client for 30 sec - exiting 13:26:19 (4824): No heartbeat from core client for 30 sec - exiting 13:26:20 (4824): No heartbeat from core client for 30 sec - exiting 13:26:21 (4824): No heartbeat from core client for 30 sec - exiting 13:26:22 (4824): No heartbeat from core client for 30 sec - exiting 13:26:23 (4824): No heartbeat from core client for 30 sec - exiting 13:26:24 (4824): No heartbeat from core client for 30 sec - exiting 13:26:25 (4824): No heartbeat from core client for 30 sec - exiting 13:26:26 (4824): No heartbeat from core client for 30 sec - exiting 13:26:27 (4824): No heartbeat from core client for 30 sec - exiting 13:26:29 (4824): No heartbeat from core client for 30 sec - exiting 13:26:30 (4824): No heartbeat from core client for 30 sec - exiting 13:26:31 (4824): No heartbeat from core client for 30 sec - exiting 13:26:32 (4824): No heartbeat from core client for 30 sec - exiting 13:26:33 (4824): No heartbeat from core client for 30 sec - exiting 13:26:34 (4824): No heartbeat from core client for 30 sec - exiting 13:26:35 (4824): No heartbeat from core client for 30 sec - exiting 13:26:36 (4824): No heartbeat from core client for 30 sec - exiting 13:26:37 (4824): No heartbeat from core client for 30 sec - exiting 13:26:38 (4824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5348, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2988, selfPID=5096, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7272, selfPID=4392, iMonCtr=1 Model crash detected, will try to restart... 09:49:31 (4616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5204, selfPID=4068, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6072, selfPID=7368, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 15:30:03 (8596): No heartbeat from core client for 30 sec - exiting 15:30:04 (8596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:30:05 (8596): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7472, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7704, selfPID=6980, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5332, selfPID=5124, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9016, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9920, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6356, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7052, selfPID=4584, iMonCtr=1 Model crash detected, will try to restart... 13:43:00 (4620): No heartbeat from core client for 30 sec - exiting 13:43:02 (4620): No heartbeat from core client for 30 sec - exiting 13:43:03 (4620): No heartbeat from core client for 30 sec - exiting 13:43:04 (4620): No heartbeat from core client for 30 sec - exiting 13:43:05 (4620): No heartbeat from core client for 30 sec - exiting 13:43:06 (4620): No heartbeat from core client for 30 sec - exiting 13:43:07 (4620): No heartbeat from core client for 30 sec - exiting 13:43:08 (4620): No heartbeat from core client for 30 sec - exiting 13:43:09 (4620): No heartbeat from core client for 30 sec - exiting 13:43:10 (4620): No heartbeat from core client for 30 sec - exiting 13:43:11 (4620): No heartbeat from core client for 30 sec - exiting 13:43:12 (4620): No heartbeat from core client for 30 sec - exiting 13:43:13 (4620): No heartbeat from core client for 30 sec - exiting 13:43:14 (4620): No heartbeat from core client for 30 sec - exiting 13:43:15 (4620): No heartbeat from core client for 30 sec - exiting 13:43:16 (4620): No heartbeat from core client for 30 sec - exiting 13:43:17 (4620): No heartbeat from core client for 30 sec - exiting 13:43:18 (4620): No heartbeat from core client for 30 sec - exiting 13:43:19 (4620): No heartbeat from core client for 30 sec - exiting 13:43:20 (4620): No heartbeat from core client for 30 sec - exiting 13:43:21 (4620): No heartbeat from core client for 30 sec - exiting 13:43:22 (4620): No heartbeat from core client for 30 sec - exiting 13:43:23 (4620): No heartbeat from core client for 30 sec - exiting 13:43:24 (4620): No heartbeat from core client for 30 sec - exiting 13:43:25 (4620): No heartbeat from core client for 30 sec - exiting 13:43:26 (4620): No heartbeat from core client for 30 sec - exiting 13:43:27 (4620): No heartbeat from core client for 30 sec - exiting 13:43:28 (4620): No heartbeat from core client for 30 sec - exiting 13:43:29 (4620): No heartbeat from core client for 30 sec - exiting 13:43:30 (4620): No heartbeat from core client for 30 sec - exiting 13:43:31 (4620): No heartbeat from core client for 30 sec - exiting 13:43:32 (4620): No heartbeat from core client for 30 sec - exiting 13:43:33 (4620): No heartbeat from core client for 30 sec - exiting 13:43:34 (4620): No heartbeat from core client for 30 sec - exiting 13:43:35 (4620): No heartbeat from core client for 30 sec - exiting 13:43:36 (4620): No heartbeat from core client for 30 sec - exiting 13:43:37 (4620): No heartbeat from core client for 30 sec - exiting 13:43:38 (4620): No heartbeat from core client for 30 sec - exiting 13:43:39 (4620): No heartbeat from core client for 30 sec - exiting 13:43:40 (4620): No heartbeat from core client for 30 sec - exiting 13:43:41 (4620): No heartbeat from core client for 30 sec - exiting 13:43:42 (4620): No heartbeat from core client for 30 sec - exiting 13:43:43 (4620): No heartbeat from core client for 30 sec - exiting 13:43:44 (4620): No heartbeat from core client for 30 sec - exiting 13:43:45 (4620): No heartbeat from core client for 30 sec - exiting 13:43:46 (4620): No heartbeat from core client for 30 sec - exiting 13:43:47 (4620): No heartbeat from core client for 30 sec - exiting 13:43:48 (4620): No heartbeat from core client for 30 sec - exiting 13:43:49 (4620): No heartbeat from core client for 30 sec - exiting 13:43:50 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5440, selfPID=5020, iMonCtr=1 Model crash detected, will try to restart... 06:27:01 (7980): No heartbeat from core client for 30 sec - exiting 06:27:02 (7980): No heartbeat from core client for 30 sec - exiting 06:27:03 (7980): No heartbeat from core client for 30 sec - exiting 06:27:04 (7980): No heartbeat from core client for 30 sec - exiting 06:27:05 (7980): No heartbeat from core client for 30 sec - exiting 06:27:06 (7980): No heartbeat from core client for 30 sec - exiting 06:27:07 (7980): No heartbeat from core client for 30 sec - exiting 06:27:08 (7980): No heartbeat from core client for 30 sec - exiting 06:27:09 (7980): No heartbeat from core client for 30 sec - exiting 06:27:10 (7980): No heartbeat from core client for 30 sec - exiting 06:27:11 (7980): No heartbeat from core client for 30 sec - exiting 06:27:13 (7980): No heartbeat from core client for 30 sec - exiting 06:27:14 (7980): No heartbeat from core client for 30 sec - exiting 06:27:15 (7980): No heartbeat from core client for 30 sec - exiting 06:27:16 (7980): No heartbeat from core client for 30 sec - exiting 06:27:17 (7980): No heartbeat from core client for 30 sec - exiting 06:27:18 (7980): No heartbeat from core client for 30 sec - exiting 06:27:19 (7980): No heartbeat from core client for 30 sec - exiting 06:27:20 (7980): No heartbeat from core client for 30 sec - exiting 06:27:21 (7980): No heartbeat from core client for 30 sec - exiting 06:27:22 (7980): No heartbeat from core client for 30 sec - exiting 06:27:23 (7980): No heartbeat from core client for 30 sec - exiting 06:27:25 (7980): No heartbeat from core client for 30 sec - exiting 06:27:26 (7980): No heartbeat from core client for 30 sec - exiting 06:27:27 (7980): No heartbeat from core client for 30 sec - exiting 06:27:28 (7980): No heartbeat from core client for 30 sec - exiting 06:27:29 (7980): No heartbeat from core client for 30 sec - exiting 06:27:30 (7980): No heartbeat from core client for 30 sec - exiting 06:27:31 (7980): No heartbeat from core client for 30 sec - exiting 06:27:32 (7980): No heartbeat from core client for 30 sec - exiting 06:27:33 (7980): No heartbeat from core client for 30 sec - exiting 06:27:34 (7980): No heartbeat from core client for 30 sec - exiting 06:27:35 (7980): No heartbeat from core client for 30 sec - exiting 06:27:37 (7980): No heartbeat from core client for 30 sec - exiting 06:27:38 (7980): No heartbeat from core client for 30 sec - exiting 06:27:39 (7980): No heartbeat from core client for 30 sec - exiting 06:27:40 (7980): No heartbeat from core client for 30 sec - exiting 06:27:41 (7980): No heartbeat from core client for 30 sec - exiting 06:27:42 (7980): No heartbeat from core client for 30 sec - exiting 06:27:43 (7980): No heartbeat from core client for 30 sec - exiting 06:27:44 (7980): No heartbeat from core client for 30 sec - exiting 06:27:45 (7980): No heartbeat from core client for 30 sec - exiting 06:27:46 (7980): No heartbeat from core client for 30 sec - exiting 06:27:48 (7980): No heartbeat from core client for 30 sec - exiting 06:27:49 (7980): No heartbeat from core client for 30 sec - exiting 06:27:50 (7980): No heartbeat from core client for 30 sec - exiting 06:27:51 (7980): No heartbeat from core client for 30 sec - exiting 06:27:52 (7980): No heartbeat from core client for 30 sec - exiting 06:27:53 (7980): No heartbeat from core client for 30 sec - exiting 06:27:54 (7980): No heartbeat from core client for 30 sec - exiting 06:27:55 (7980): No heartbeat from core client for 30 sec - exiting 06:27:56 (7980): No heartbeat from core client for 30 sec - exiting 06:27:57 (7980): No heartbeat from core client for 30 sec - exiting 06:27:58 (7980): No heartbeat from core client for 30 sec - exiting 06:28:00 (7980): No heartbeat from core client for 30 sec - exiting 06:28:01 (7980): No heartbeat from core client for 30 sec - exiting 06:28:02 (7980): No heartbeat from core client for 30 sec - exiting 06:28:03 (7980): No heartbeat from core client for 30 sec - exiting 06:28:04 (7980): No heartbeat from core client for 30 sec - exiting 06:28:05 (7980): No heartbeat from core client for 30 sec - exiting 06:28:06 (7980): No heartbeat from core client for 30 sec - exiting 06:28:07 (7980): No heartbeat from core client for 30 sec - exiting 06:28:08 (7980): No heartbeat from core client for 30 sec - exiting 06:28:09 (7980): No heartbeat from core client for 30 sec - exiting 06:28:10 (7980): No heartbeat from core client for 30 sec - exiting 06:28:12 (7980): No heartbeat from core client for 30 sec - exiting 06:28:13 (7980): No heartbeat from core client for 30 sec - exiting 06:28:14 (7980): No heartbeat from core client for 30 sec - exiting 06:28:15 (7980): No heartbeat from core client for 30 sec - exiting 06:28:16 (7980): No heartbeat from core client for 30 sec - exiting 06:28:17 (7980): No heartbeat from core client for 30 sec - exiting 06:28:18 (7980): No heartbeat from core client for 30 sec - exiting 06:28:19 (7980): No heartbeat from core client for 30 sec - exiting 06:28:20 (7980): No heartbeat from core client for 30 sec - exiting 06:28:21 (7980): No heartbeat from core client for 30 sec - exiting 06:28:22 (7980): No heartbeat from core client for 30 sec - exiting 06:28:24 (7980): No heartbeat from core client for 30 sec - exiting 06:28:25 (7980): No heartbeat from core client for 30 sec - exiting 06:28:26 (7980): No heartbeat from core client for 30 sec - exiting 06:28:27 (7980): No heartbeat from core client for 30 sec - exiting 06:28:28 (7980): No heartbeat from core client for 30 sec - exiting 06:28:29 (7980): No heartbeat from core client for 30 sec - exiting 06:28:30 (7980): No heartbeat from core client for 30 sec - exiting 06:28:31 (7980): No heartbeat from core client for 30 sec - exiting 06:28:32 (7980): No heartbeat from core client for 30 sec - exiting 06:28:33 (7980): No heartbeat from core client for 30 sec - exiting 06:28:34 (7980): No heartbeat from core client for 30 sec - exiting 06:28:36 (7980): No heartbeat from core client for 30 sec - exiting 06:28:37 (7980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9328, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9120, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=7892, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_o8dh_2012_1_008627635_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Apr 2014 15:58:45	1322841	16459025	hadam3p_anz_o8dh_2012_1_008627635_0	34,859	258,951	7.4285
24 Apr 2014 22:05:46	1322841	16459025	hadam3p_anz_o8dh_2012_1_008627635_0	23,339	180,099	7.7167
17 Apr 2014 16:15:39	1322841	16459025	hadam3p_anz_o8dh_2012_1_008627635_0	11,819	90,485	7.6559