Task 15098960

Name	hadam3p_eu_8bb3_2000_1_008132731_0
Workunit	8287845
Created	11 Aug 2012, 23:24:31 UTC
Sent	11 Aug 2012, 23:28:10 UTC
Report deadline	25 Jul 2013, 4:48:10 UTC
Received	3 Sep 2012, 4:06:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1145373
Run time	5 days 23 hours 55 min 49 sec
CPU time	5 days 6 hours 17 min 59 sec
Validate state	Invalid
Credit	1,988.94
Device peak FLOPS	1.70 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:41:01 (3122): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:36:55 (4761): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:11:50 (6653): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:28 (6773): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:29:31 (1536): No heartbeat from core client for 30 sec - exiting 03:29:32 (1536): No heartbeat from core client for 30 sec - exiting 03:29:33 (1536): No heartbeat from core client for 30 sec - exiting 03:29:34 (1536): No heartbeat from core client for 30 sec - exiting 03:29:38 (1536): No heartbeat from core client for 30 sec - exiting 03:29:39 (1536): No heartbeat from core client for 30 sec - exiting 03:29:40 (1536): No heartbeat from core client for 30 sec - exiting 03:29:41 (1536): No heartbeat from core client for 30 sec - exiting 03:29:44 (1536): No heartbeat from core client for 30 sec - exiting 03:29:45 (1536): No heartbeat from core client for 30 sec - exiting 03:29:46 (1536): No heartbeat from core client for 30 sec - exiting 03:29:47 (1536): No heartbeat from core client for 30 sec - exiting 03:29:48 (1536): No heartbeat from core client for 30 sec - exiting 03:29:49 (1536): No heartbeat from core client for 30 sec - exiting 03:29:56 (1536): No heartbeat from core client for 30 sec - exiting 03:29:59 (1536): No heartbeat from core client for 30 sec - exiting 03:30:00 (1536): No heartbeat from core client for 30 sec - exiting 03:30:01 (1536): No heartbeat from core client for 30 sec - exiting 03:30:02 (1536): No heartbeat from core client for 30 sec - exiting 03:30:03 (1536): No heartbeat from core client for 30 sec - exiting 03:30:04 (1536): No heartbeat from core client for 30 sec - exiting 03:30:05 (1536): No heartbeat from core client for 30 sec - exiting 03:30:06 (1536): No heartbeat from core client for 30 sec - exiting 03:30:13 (1536): No heartbeat from core client for 30 sec - exiting 03:30:14 (1536): No heartbeat from core client for 30 sec - exiting 03:30:15 (1536): No heartbeat from core client for 30 sec - exiting 03:30:16 (1536): No heartbeat from core client for 30 sec - exiting 03:30:17 (1536): No heartbeat from core client for 30 sec - exiting 03:30:18 (1536): No heartbeat from core client for 30 sec - exiting 03:30:19 (1536): No heartbeat from core client for 30 sec - exiting 03:30:20 (1536): No heartbeat from core client for 30 sec - exiting 03:30:21 (1536): No heartbeat from core client for 30 sec - exiting 03:30:22 (1536): No heartbeat from core client for 30 sec - exiting 03:30:23 (1536): No heartbeat from core client for 30 sec - exiting 03:30:24 (1536): No heartbeat from core client for 30 sec - exiting 03:30:25 (1536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6649, selfPID=6650, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:14:00 (1595): No heartbeat from core client for 30 sec - exiting 05:14:01 (1595): No heartbeat from core client for 30 sec - exiting 05:14:02 (1595): No heartbeat from core client for 30 sec - exiting 05:14:03 (1595): No heartbeat from core client for 30 sec - exiting 05:14:04 (1595): No heartbeat from core client for 30 sec - exiting 05:14:05 (1595): No heartbeat from core client for 30 sec - exiting 05:14:06 (1595): No heartbeat from core client for 30 sec - exiting 05:14:07 (1595): No heartbeat from core client for 30 sec - exiting 05:14:08 (1595): No heartbeat from core client for 30 sec - exiting 05:14:15 (1595): No heartbeat from core client for 30 sec - exiting 05:14:16 (1595): No heartbeat from core client for 30 sec - exiting 05:14:17 (1595): No heartbeat from core client for 30 sec - exiting 05:14:21 (1595): No heartbeat from core client for 30 sec - exiting 05:14:22 (1595): No heartbeat from core client for 30 sec - exiting 05:14:23 (1595): No heartbeat from core client for 30 sec - exiting 05:14:29 (1595): No heartbeat from core client for 30 sec - exiting 05:14:36 (1595): No heartbeat from core client for 30 sec - exiting 05:14:37 (1595): No heartbeat from core client for 30 sec - exiting 05:14:38 (1595): No heartbeat from core client for 30 sec - exiting 05:14:41 (1595): No heartbeat from core client for 30 sec - exiting 05:14:42 (1595): No heartbeat from core client for 30 sec - exiting 05:14:43 (1595): No heartbeat from core client for 30 sec - exiting 05:14:45 (1595): No heartbeat from core client for 30 sec - exiting 05:14:46 (1595): No heartbeat from core client for 30 sec - exiting 05:14:47 (1595): No heartbeat from core client for 30 sec - exiting 05:14:48 (1595): No heartbeat from core client for 30 sec - exiting 05:14:49 (1595): No heartbeat from core client for 30 sec - exiting 05:14:50 (1595): No heartbeat from core client for 30 sec - exiting 05:14:51 (1595): No heartbeat from core client for 30 sec - exiting 05:14:52 (1595): No heartbeat from core client for 30 sec - exiting 05:14:53 (1595): No heartbeat from core client for 30 sec - exiting 05:14:54 (1595): No heartbeat from core client for 30 sec - exiting 05:14:55 (1595): No heartbeat from core client for 30 sec - exiting 05:14:56 (1595): No heartbeat from core client for 30 sec - exiting 05:14:57 (1595): No heartbeat from core client for 30 sec - exiting 05:14:58 (1595): No heartbeat from core client for 30 sec - exiting 05:14:59 (1595): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5583, iMonCtr=2 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish Leaving CPDN_Main::Monitor... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_eu_8bb3_2000_1_008132731_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_8bb3_2000_1_008132731_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_8bb3_2000_1_008132731_0_13.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Aug 2012 16:27:36	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	115,296	420,156	3.6442
29 Aug 2012 13:31:59	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	103,776	378,391	3.6462
27 Aug 2012 17:13:54	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	92,256	336,620	3.6488
25 Aug 2012 04:38:54	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	80,736	294,553	3.6483
23 Aug 2012 15:32:52	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	69,216	253,025	3.6556
21 Aug 2012 11:40:36	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	57,696	210,849	3.6545
19 Aug 2012 12:15:01	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	46,176	168,456	3.6481
17 Aug 2012 16:01:30	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	34,656	126,413	3.6477
15 Aug 2012 21:58:12	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	23,136	84,485	3.6517
14 Aug 2012 06:00:58	1145373	15098960	hadam3p_eu_8bb3_2000_1_008132731_0	11,616	42,495	3.6583