Task 16823576

Name	hadam3p_eu_o4we_2013_1_008832935_1
Workunit	8978864
Created	25 Jul 2014, 8:06:16 UTC
Sent	25 Jul 2014, 8:13:22 UTC
Report deadline	7 Jul 2015, 13:33:22 UTC
Received	10 Nov 2014, 20:55:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1147232
Run time	6 days 3 hours 0 min 39 sec
CPU time	2 hours 11 min 38 sec
Validate state	Invalid
Credit	2,187.87
Device peak FLOPS	2.10 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2660, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8812, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3084, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5724, selfPID=344, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9384, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=46552, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3720, selfPID=5604, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5472, iMonCtr=2 Model crash detected, will try to restart... GSuspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5076, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15536, iMonCtr=2 CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5348, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3800, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5020, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4220, selfPID=4004, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5276, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4896, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3964, selfPID=5256, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 18:53:46 (6576): No heartbeat from core client for 30 sec - exiting 18:53:47 (6576): No heartbeat from core client for 30 sec - exiting 18:53:48 (6576): No heartbeat from core client for 30 sec - exiting 18:53:49 (6576): No heartbeat from core client for 30 sec - exiting 18:53:50 (6576): No heartbeat from core client for 30 sec - exiting 18:53:51 (6576): No heartbeat from core client for 30 sec - exiting 18:53:52 (6576): No heartbeat from core client for 30 sec - exiting 18:53:53 (6576): No heartbeat from core client for 30 sec - exiting 18:53:54 (6576): No heartbeat from core client for 30 sec - exiting 18:53:55 (6576): No heartbeat from core client for 30 sec - exiting 18:53:56 (6576): No heartbeat from core client for 30 sec - exiting 18:53:57 (6576): No heartbeat from core client for 30 sec - exiting 18:53:58 (6576): No heartbeat from core client for 30 sec - exiting 18:54:00 (6576): No heartbeat from core client for 30 sec - exiting 18:54:01 (6576): No heartbeat from core client for 30 sec - exiting 18:54:02 (6576): No heartbeat from core client for 30 sec - exiting 18:54:03 (6576): No heartbeat from core client for 30 sec - exiting 18:54:04 (6576): No heartbeat from core client for 30 sec - exiting 18:54:05 (6576): No heartbeat from core client for 30 sec - exiting 18:54:06 (6576): No heartbeat from core client for 30 sec - exiting 18:54:07 (6576): No heartbeat from core client for 30 sec - exiting 18:54:08 (6576): No heartbeat from core client for 30 sec - exiting 18:54:09 (6576): No heartbeat from core client for 30 sec - exiting 18:54:10 (6576): No heartbeat from core client for 30 sec - exiting 18:54:11 (6576): No heartbeat from core client for 30 sec - exiting 18:54:12 (6576): No heartbeat from core client for 30 sec - exiting 18:54:13 (6576): No heartbeat from core client for 30 sec - exiting 18:54:14 (6576): No heartbeat from core client for 30 sec - exiting 18:54:15 (6576): No heartbeat from core client for 30 sec - exiting 18:54:16 (6576): No heartbeat from core client for 30 sec - exiting 18:54:17 (6576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7608, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7616, selfPID=7592, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2444, selfPID=4672, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6028, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadam3p_eu_o4we_2013_1_008832935/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadam3p_eu_o4we_2013_1_008832935/dataout/region_restart.day after 11 attempts Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_eu_o4we_2013_1_008832935_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Nov 2014 16:10:45	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	126,828	382,083	3.0126
25 Oct 2014 11:54:05	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	126,816	381,599	3.0091
15 Oct 2014 20:07:00	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	115,296	347,929	3.0177
08 Oct 2014 22:16:55	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	103,776	313,677	3.0226
30 Sep 2014 16:27:03	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	92,264	279,836	3.0330
30 Sep 2014 14:30:04	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	92,256	279,369	3.0282
30 Sep 2014 04:30:31	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	80,736	246,677	3.0554
29 Sep 2014 16:29:11	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	69,216	213,501	3.0846
27 Sep 2014 01:32:53	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	57,706	180,026	3.1197
27 Sep 2014 00:31:55	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	57,696	179,556	3.1121
29 Aug 2014 13:19:31	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	46,176	145,381	3.1484
14 Aug 2014 14:35:16	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	34,656	109,378	3.1561
14 Aug 2014 14:35:16	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	23,147	73,577	3.1787
14 Aug 2014 14:35:16	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	23,136	73,049	3.1574
14 Aug 2014 14:35:16	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	23,136	73,049	3.1574
06 Aug 2014 10:20:54	1147232	16823576	hadam3p_eu_o4we_2013_1_008832935_1	11,616	36,305	3.1254