Task 13204333

Name	hadam3p_pnw_3025_1985_1_007390859_0
Workunit	7588289
Created	6 Aug 2011, 2:57:38 UTC
Sent	6 Aug 2011, 2:57:42 UTC
Report deadline	18 Jul 2012, 8:17:42 UTC
Received	22 Nov 2011, 3:38:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1109502
Run time	5 days 1 hours 41 min 26 sec
CPU time	4 days 7 hours 15 min 45 sec
Validate state	Invalid
Credit	1,754.30
Device peak FLOPS	1.80 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5032, selfPID=5032, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5052, selfPID=5052, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3604, selfPID=3604, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7120, selfPID=7120, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6524, selfPID=6524, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2664, selfPID=2664, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7972, selfPID=7972, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=312, selfPID=312, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6376, selfPID=6376, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5536, selfPID=5536, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... RCPDN Monitor - Quit request from BOINC... 08:37:13 (8264): No heartbeat from core client for 30 sec - exiting 08:37:14 (8264): No heartbeat from core client for 30 sec - exiting 08:37:15 (8264): No heartbeat from core client for 30 sec - exiting 08:37:16 (8264): No heartbeat from core client for 30 sec - exiting 08:37:17 (8264): No heartbeat from core client for 30 sec - exiting 08:37:18 (8264): No heartbeat from core client for 30 sec - exiting 08:37:19 (8264): No heartbeat from core client for 30 sec - exiting 08:37:20 (8264): No heartbeat from core client for 30 sec - exiting 08:37:21 (8264): No heartbeat from core client for 30 sec - exiting 08:37:22 (8264): No heartbeat from core client for 30 sec - exiting 08:37:23 (8264): No heartbeat from core client for 30 sec - exiting 08:37:24 (8264): No heartbeat from core client for 30 sec - exiting 08:37:25 (8264): No heartbeat from core client for 30 sec - exiting 08:37:26 (8264): No heartbeat from core client for 30 sec - exiting 08:37:27 (8264): No heartbeat from core client for 30 sec - exiting 08:37:28 (8264): No heartbeat from core client for 30 sec - exiting 08:37:29 (8264): No heartbeat from core client for 30 sec - exiting 08:37:30 (8264): No heartbeat from core client for 30 sec - exiting 08:37:31 (8264): No heartbeat from core client for 30 sec - exiting 08:37:32 (8264): No heartbeat from core client for 30 sec - exiting 08:37:33 (8264): No heartbeat from core client for 30 sec - exiting 08:37:34 (8264): No heartbeat from core client for 30 sec - exiting 08:37:35 (8264): No heartbeat from core client for 30 sec - exiting 08:37:36 (8264): No heartbeat from core client for 30 sec - exiting 08:37:37 (8264): No heartbeat from core client for 30 sec - exiting 08:37:38 (8264): No heartbeat from core client for 30 sec - exiting 08:37:39 (8264): No heartbeat from core client for 30 sec - exiting 08:37:40 (8264): No heartbeat from core client for 30 sec - exiting 08:37:41 (8264): No heartbeat from core client for 30 sec - exiting 08:37:42 (8264): No heartbeat from core client for 30 sec - exiting 08:37:43 (8264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:37:44 (8264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5868, selfPID=5868, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3784, selfPID=3784, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7900, selfPID=7900, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4912, selfPID=4912, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:29:02 (7664): No heartbeat from core client for 30 sec - exiting 17:29:03 (7664): No heartbeat from core client for 30 sec - exiting 17:29:04 (7664): No heartbeat from core client for 30 sec - exiting 17:29:05 (7664): No heartbeat from core client for 30 sec - exiting 17:29:06 (7664): No heartbeat from core client for 30 sec - exiting 17:29:07 (7664): No heartbeat from core client for 30 sec - exiting 17:29:08 (7664): No heartbeat from core client for 30 sec - exiting 17:29:09 (7664): No heartbeat from core client for 30 sec - exiting 17:29:10 (7664): No heartbeat from core client for 30 sec - exiting 17:29:11 (7664): No heartbeat from core client for 30 sec - exiting 17:29:12 (7664): No heartbeat from core client for 30 sec - exiting 17:29:13 (7664): No heartbeat from core client for 30 sec - exiting 17:29:14 (7664): No heartbeat from core client for 30 sec - exiting 17:29:15 (7664): No heartbeat from core client for 30 sec - exiting 17:29:16 (7664): No heartbeat from core client for 30 sec - exiting 17:29:17 (7664): No heartbeat from core client for 30 sec - exiting 17:29:18 (7664): No heartbeat from core client for 30 sec - exiting 17:29:19 (7664): No heartbeat from core client for 30 sec - exiting 17:29:20 (7664): No heartbeat from core client for 30 sec - exiting 17:29:21 (7664): No heartbeat from core client for 30 sec - exiting 17:29:22 (7664): No heartbeat from core client for 30 sec - exiting 17:29:23 (7664): No heartbeat from core client for 30 sec - exiting 17:29:24 (7664): No heartbeat from core client for 30 sec - exiting 17:29:25 (7664): No heartbeat from core client for 30 sec - exiting 17:29:26 (7664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5356, selfPID=5356, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5028, selfPID=5028, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2032, selfPID=2032, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6284, selfPID=6284, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2900, selfPID=3888, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4552, selfPID=4552, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4616, selfPID=4616, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2788, selfPID=2788, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5708, selfPID=5708, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2872, selfPID=2872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4148, selfPID=4148, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4104, selfPID=4104, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=704, selfPID=704, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6908, selfPID=6908, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4916, selfPID=4916, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:51:07 (6348): No heartbeat from core client for 30 sec - exiting 16:51:08 (6348): No heartbeat from core client for 30 sec - exiting 16:51:09 (6348): No heartbeat from core client for 30 sec - exiting 16:51:10 (6348): No heartbeat from core client for 30 sec - exiting 16:51:12 (6348): No heartbeat from core client for 30 sec - exiting 16:51:13 (6348): No heartbeat from core client for 30 sec - exiting 16:51:14 (6348): No heartbeat from core client for 30 sec - exiting 16:51:15 (6348): No heartbeat from core client for 30 sec - exiting 16:51:16 (6348): No heartbeat from core client for 30 sec - exiting 16:51:17 (6348): No heartbeat from core client for 30 sec - exiting 16:51:18 (6348): No heartbeat from core client for 30 sec - exiting 16:51:19 (6348): No heartbeat from core client for 30 sec - exiting 16:51:20 (6348): No heartbeat from core client for 30 sec - exiting 16:51:21 (6348): No heartbeat from core client for 30 sec - exiting 16:51:22 (6348): No heartbeat from core client for 30 sec - exiting 16:51:23 (6348): No heartbeat from core client for 30 sec - exiting 16:51:24 (6348): No heartbeat from core client for 30 sec - exiting 16:51:25 (6348): No heartbeat from core client for 30 sec - exiting 16:51:26 (6348): No heartbeat from core client for 30 sec - exiting 16:51:27 (6348): No heartbeat from core client for 30 sec - exiting 16:51:28 (6348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6644, selfPID=10060, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4600, selfPID=4600, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3260, selfPID=3260, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8924, selfPID=8924, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9224, selfPID=9224, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8168, selfPID=8168, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exitibRetVal = 1, checkPID=204, selfPID=2484, iMonCtr=1 Model crash detected, will try to restart... ng, bRetVal = 1, checkPID=204, selfPID=204, iMonCtr=2 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_3025_1985_1_007390859_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_3025_1985_1_007390859_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_3025_1985_1_007390859_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_3025_1985_1_007390859_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_3025_1985_1_007390859_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2011 16:00:33	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	80,736	358,511	4.4405
03 Nov 2011 16:06:52	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	69,216	310,706	4.4889
17 Oct 2011 12:24:57	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	57,696	259,856	4.5039
09 Sep 2011 20:56:59	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	46,176	208,404	4.5133
03 Sep 2011 23:25:33	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	34,656	156,716	4.5220
19 Aug 2011 12:42:13	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	23,136	106,198	4.5902
16 Aug 2011 09:14:49	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	11,618	55,660	4.7908
15 Aug 2011 23:11:07	1109502	13204333	hadam3p_pnw_3025_1985_1_007390859_0	11,616	55,053	4.7394