Task 17549778

Name	hadam3p_pnw_hcw5_2011_1_009287467_0
Workunit	9371655
Created	4 Dec 2014, 11:38:35 UTC
Sent	4 Dec 2014, 17:30:19 UTC
Report deadline	16 Nov 2015, 22:50:19 UTC
Received	9 Mar 2015, 19:58:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1272486
Run time	4 days 18 hours 47 min 50 sec
CPU time	4 days 9 hours 47 min 53 sec
Validate state	Invalid
Credit	2,759.97
Device peak FLOPS	2.90 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v7.22 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> 18:07:21 (3884): No heartbeat from client for 30 sec - exiting 18:07:21 (3884): timer handler: client dead, exiting 18:07:22 (3884): No heartbeat from client for 30 sec - exiting 18:07:22 (3884): timer handler: client dead, exiting 18:07:23 (3884): No heartbeat from client for 30 sec - exiting 18:07:23 (3884): timer handler: client dead, exiting 18:07:24 (3884): No heartbeat from client for 30 sec - exiting 18:07:24 (3884): timer handler: client dead, exiting 18:07:25 (3884): No heartbeat from client for 30 sec - exiting 18:07:25 (3884): timer handler: client dead, exiting 18:07:26 (3884): No heartbeat from client for 30 sec - exiting 18:07:26 (3884): timer handler: client dead, exiting 18:07:27 (3884): No heartbeat from client for 30 sec - exiting 18:07:27 (3884): timer handler: client dead, exiting 18:07:28 (3884): No heartbeat from client for 30 sec - exiting 18:07:28 (3884): timer handler: client dead, exiting 18:07:30 (3884): No heartbeat from client for 30 sec - exiting 18:07:30 (3884): timer handler: client dead, exiting 18:07:31 (3884): No heartbeat from client for 30 sec - exiting 18:07:31 (3884): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3784, selfPID=2936, iMonCtr=1 Model crash detected, will try to restart... 11:09:05 (3644): No heartbeat from client for 30 sec - exiting 11:09:05 (3644): timer handler: client dead, exiting 11:09:06 (3644): No heartbeat from client for 30 sec - exiting 11:09:06 (3644): timer handler: client dead, exiting 11:09:07 (3644): No heartbeat from client for 30 sec - exiting 11:09:07 (3644): timer handler: client dead, exiting 11:09:08 (3644): No heartbeat from client for 30 sec - exiting 11:09:08 (3644): timer handler: client dead, exiting 11:09:10 (3644): No heartbeat from client for 30 sec - exiting 11:09:10 (3644): timer handler: client dead, exiting 11:09:11 (3644): No heartbeat from client for 30 sec - exiting 11:09:11 (3644): timer handler: client dead, exiting 11:09:12 (3644): No heartbeat from client for 30 sec - exiting 11:09:12 (3644): timer handler: client dead, exiting 11:09:13 (3644): No heartbeat from client for 30 sec - exiting 11:09:13 (3644): timer handler: client dead, exiting 11:09:14 (3644): No heartbeat from client for 30 sec - exiting 11:09:14 (3644): timer handler: client dead, exiting 11:09:15 (3644): No heartbeat from client for 30 sec - exiting 11:09:15 (3644): timer handler: client dead, exiting 11:09:16 (3644): No heartbeat from client for 30 sec - exiting 11:09:16 (3644): timer handler: client dead, exiting 11:09:17 (3644): No heartbeat from client for 30 sec - exiting 11:09:17 (3644): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CGlobal Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3488, iMonCtr=2 GSuspended CPDN Monitor - Suspend request from BOINC... 09:28:29 (5204): No heartbeat from client for 30 sec - exiting 09:28:29 (5204): timer handler: client dead, exiting 09:28:30 (5204): No heartbeat from client for 30 sec - exiting 09:28:30 (5204): timer handler: client dead, exiting 09:28:31 (5204): No heartbeat from client for 30 sec - exiting 09:28:31 (5204): timer handler: client dead, exiting 09:28:32 (5204): No heartbeat from client for 30 sec - exiting 09:28:32 (5204): timer handler: client dead, exiting 09:28:33 (5204): No heartbeat from client for 30 sec - exiting 09:28:33 (5204): timer handler: client dead, exiting 09:28:34 (5204): No heartbeat from client for 30 sec - exiting 09:28:34 (5204): timer handler: client dead, exiting 09:28:35 (5204): No heartbeat from client for 30 sec - exiting 09:28:35 (5204): timer handler: client dead, exiting 09:28:36 (5204): No heartbeat from client for 30 sec - exiting 09:28:36 (5204): timer handler: client dead, exiting 09:28:37 (5204): No heartbeat from client for 30 sec - exiting 09:28:37 (5204): timer handler: client dead, exiting 09:28:38 (5204): No heartbeat from client for 30 sec - exiting 09:28:38 (5204): timer handler: client dead, exiting 09:28:39 (5204): No heartbeat from client for 30 sec - exiting 09:28:39 (5204): timer handler: client dead, exiting 09:28:40 (5204): No heartbeat from client for 30 sec - exiting 09:28:40 (5204): timer handler: client dead, exiting 09:28:41 (5204): No heartbeat from client for 30 sec - exiting 09:28:41 (5204): timer handler: client dead, exiting 09:28:42 (5204): No heartbeat from client for 30 sec - exiting 09:28:42 (5204): timer handler: client dead, exiting 09:28:43 (5204): No heartbeat from client for 30 sec - exiting 09:28:43 (5204): timer handler: client dead, exiting 09:28:44 (5204): No heartbeat from client for 30 sec - exiting 09:28:44 (5204): timer handler: client dead, exiting 09:28:45 (5204): No heartbeat from client for 30 sec - exiting 09:28:45 (5204): timer handler: client dead, exiting 09:28:46 (5204): No heartbeat from client for 30 sec - exiting 09:28:46 (5204): timer handler: client dead, exiting 09:28:47 (5204): No heartbeat from client for 30 sec - exiting 09:28:47 (5204): timer handler: client dead, exiting 09:28:48 (5204): No heartbeat from client for 30 sec - exiting 09:28:48 (5204): timer handler: client dead, exiting 09:28:49 (5204): No heartbeat from client for 30 sec - exiting 09:28:49 (5204): timer handler: client dead, exiting 09:28:50 (5204): No heartbeat from client for 30 sec - exiting 09:28:50 (5204): timer handler: client dead, exiting 09:28:51 (5204): No heartbeat from client for 30 sec - exiting 09:28:51 (5204): timer handler: client dead, exiting 09:28:52 (5204): No heartbeat from client for 30 sec - exiting 09:28:52 (5204): timer handler: client dead, exiting 09:28:53 (5204): No heartbeat from client for 30 sec - exiting 09:28:53 (5204): timer handler: client dead, exiting 09:28:54 (5204): No heartbeat from client for 30 sec - exiting 09:28:54 (5204): timer handler: client dead, exiting 09:28:55 (5204): No heartbeat from client for 30 sec - exiting 09:28:55 (5204): timer handler: client dead, exiting 09:28:56 (5204): No heartbeat from client for 30 sec - exiting 09:28:56 (5204): timer handler: client dead, exiting 09:28:57 (5204): No heartbeat from client for 30 sec - exiting 09:28:57 (5204): timer handler: client dead, exiting 09:28:58 (5204): No heartbeat from client for 30 sec - exiting 09:28:58 (5204): timer handler: client dead, exiting 09:28:59 (5204): No heartbeat from client for 30 sec - exiting 09:28:59 (5204): timer handler: client dead, exiting 09:29:00 (5204): No heartbeat from client for 30 sec - exiting 09:29:00 (5204): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:29:01 (5204): No heartbeat from client for 30 sec - exiting 09:29:01 (5204): timer handler: client dead, exiting 19:40:38 (2312): No heartbeat from client for 30 sec - exiting 19:40:38 (2312): timer handler: client dead, exiting 19:40:39 (2312): No heartbeat from client for 30 sec - exiting 19:40:39 (2312): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=216, selfPID=3572, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 17:55:47 (4744): No heartbeat from client for 30 sec - exiting 17:55:47 (4744): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:50:36 (3128): No heartbeat from client for 30 sec - exiting 17:50:36 (3128): timer handler: client dead, exiting 17:50:37 (3128): No heartbeat from client for 30 sec - exiting 17:50:37 (3128): timer handler: client dead, exiting 17:50:38 (3128): No heartbeat from client for 30 sec - exiting 17:50:38 (3128): timer handler: client dead, exiting 17:50:40 (3128): No heartbeat from client for 30 sec - exiting 17:50:40 (3128): timer handler: client dead, exiting 17:50:41 (3128): No heartbeat from client for 30 sec - exiting 17:50:41 (3128): timer handler: client dead, exiting 17:50:42 (3128): No heartbeat from client for 30 sec - exiting 17:50:42 (3128): timer handler: client dead, exiting 17:50:43 (3128): No heartbeat from client for 30 sec - exiting 17:50:43 (3128): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1040, selfPID=3712, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 10:15:30 (3816): No heartbeat from client for 30 sec - exiting 10:15:30 (3816): timer handler: client dead, exiting 10:15:31 (3816): No heartbeat from client for 30 sec - exiting 10:15:31 (3816): timer handler: client dead, exiting 10:15:32 (3816): No heartbeat from client for 30 sec - exiting 10:15:32 (3816): timer handler: client dead, exiting 10:15:33 (3816): No heartbeat from client for 30 sec - exiting 10:15:33 (3816): timer handler: client dead, exiting 10:15:34 (3816): No heartbeat from client for 30 sec - exiting 10:15:34 (3816): timer handler: client dead, exiting 10:15:35 (3816): No heartbeat from client for 30 sec - exiting 10:15:35 (3816): timer handler: client dead, exiting 10:15:36 (3816): No heartbeat from client for 30 sec - exiting 10:15:36 (3816): timer handler: client dead, exiting 10:15:37 (3816): No heartbeat from client for 30 sec - exiting 10:15:37 (3816): timer handler: client dead, exiting 10:15:38 (3816): No heartbeat from client for 30 sec - exiting 10:15:38 (3816): timer handler: client dead, exiting 10:15:40 (3816): No heartbeat from client for 30 sec - exiting 10:15:40 (3816): timer handler: client dead, exiting 10:15:41 (3816): No heartbeat from client for 30 sec - exiting 10:15:41 (3816): timer handler: client dead, exiting 10:15:42 (3816): No heartbeat from client for 30 sec - exiting 10:15:42 (3816): timer handler: client dead, exiting 10:15:43 (3816): No heartbeat from client for 30 sec - exiting 10:15:43 (3816): timer handler: client dead, exiting 10:15:44 (3816): No heartbeat from client for 30 sec - exiting 10:15:44 (3816): timer handler: client dead, exiting 10:15:45 (3816): No heartbeat from client for 30 sec - exiting 10:15:45 (3816): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3068, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=3848, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4788, selfPID=4660, iMonCtr=1 Model crash detected, will try to restart... 09:30:41 (3748): No heartbeat from client for 30 sec - exiting 09:30:41 (3748): timer handler: client dead, exiting 09:30:42 (3748): No heartbeat from client for 30 sec - exiting 09:30:42 (3748): timer handler: client dead, exiting 09:30:43 (3748): No heartbeat from client for 30 sec - exiting 09:30:43 (3748): timer handler: client dead, exiting 09:30:44 (3748): No heartbeat from client for 30 sec - exiting 09:30:44 (3748): timer handler: client dead, exiting 09:30:46 (3748): No heartbeat from client for 30 sec - exiting 09:30:46 (3748): timer handler: client dead, exiting 09:30:47 (3748): No heartbeat from client for 30 sec - exiting 09:30:47 (3748): timer handler: client dead, exiting 09:30:48 (3748): No heartbeat from client for 30 sec - exiting 09:30:48 (3748): timer handler: client dead, exiting 09:30:49 (3748): No heartbeat from client for 30 sec - exiting 09:30:49 (3748): timer handler: client dead, exiting 09:30:50 (3748): No heartbeat from client for 30 sec - exiting 09:30:50 (3748): timer handler: client dead, exiting 09:30:51 (3748): No heartbeat from client for 30 sec - exiting 09:30:51 (3748): timer handler: client dead, exiting 09:30:52 (3748): No heartbeat from client for 30 sec - exiting 09:30:52 (3748): timer handler: client dead, exiting 09:30:53 (3748): No heartbeat from client for 30 sec - exiting 09:30:53 (3748): timer handler: client dead, exiting 09:30:54 (3748): No heartbeat from client for 30 sec - exiting 09:30:54 (3748): timer handler: client dead, exiting 09:30:55 (3748): No heartbeat from client for 30 sec - exiting 09:30:55 (3748): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:59:24 (3932): No heartbeat from client for 30 sec - exiting 09:59:24 (3932): timer handler: client dead, exiting 09:59:25 (3932): No heartbeat from client for 30 sec - exiting 09:59:25 (3932): timer handler: client dead, exiting 09:59:26 (3932): No heartbeat from client for 30 sec - exiting 09:59:26 (3932): timer handler: client dead, exiting 09:59:27 (3932): No heartbeat from client for 30 sec - exiting 09:59:27 (3932): timer handler: client dead, exiting 09:59:28 (3932): No heartbeat from client for 30 sec - exiting 09:59:28 (3932): timer handler: client dead, exiting 09:59:29 (3932): No heartbeat from client for 30 sec - exiting 09:59:29 (3932): timer handler: client dead, exiting 09:59:31 (3932): No heartbeat from client for 30 sec - exiting 09:59:31 (3932): timer handler: client dead, exiting 09:59:32 (3932): No heartbeat from client for 30 sec - exiting 09:59:32 (3932): timer handler: client dead, exiting 09:59:33 (3932): No heartbeat from client for 30 sec - exiting 09:59:33 (3932): timer handler: client dead, exiting 09:59:34 (3932): No heartbeat from client for 30 sec - exiting 09:59:34 (3932): timer handler: client dead, exiting 09:59:35 (3932): No heartbeat from client for 30 sec - exiting 09:59:35 (3932): timer handler: client dead, exiting 09:59:36 (3932): No heartbeat from client for 30 sec - exiting 09:59:36 (3932): timer handler: client dead, exiting 09:59:37 (3932): No heartbeat from client for 30 sec - exiting 09:59:37 (3932): timer handler: client dead, exiting 09:59:38 (3932): No heartbeat from client for 30 sec - exiting 09:59:38 (3932): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5624, selfPID=5352, iMonCtr=1 Model crash detected, will try to restart... 13:13:13 (2420): No heartbeat from client for 30 sec - exiting 13:13:13 (2420): timer handler: client dead, exiting 13:13:14 (2420): No heartbeat from client for 30 sec - exiting 13:13:14 (2420): timer handler: client dead, exiting 13:13:15 (2420): No heartbeat from client for 30 sec - exiting 13:13:15 (2420): timer handler: client dead, exiting 13:13:16 (2420): No heartbeat from client for 30 sec - exiting 13:13:16 (2420): timer handler: client dead, exiting 13:13:17 (2420): No heartbeat from client for 30 sec - exiting 13:13:17 (2420): timer handler: client dead, exiting 13:13:18 (2420): No heartbeat from client for 30 sec - exiting 13:13:18 (2420): timer handler: client dead, exiting 13:13:20 (2420): No heartbeat from client for 30 sec - exiting 13:13:20 (2420): timer handler: client dead, exiting 13:13:21 (2420): No heartbeat from client for 30 sec - exiting 13:13:21 (2420): timer handler: client dead, exiting 13:13:22 (2420): No heartbeat from client for 30 sec - exiting 13:13:22 (2420): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5072, selfPID=4912, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5348, iMonCtr=2 Model crash detected, will try to restart... 15:13:16 (4960): No heartbeat from client for 30 sec - exiting 15:13:16 (4960): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:13:55 (5832): No heartbeat from client for 30 sec - exiting 15:13:55 (5832): timer handler: client dead, exiting 15:13:57 (5832): No heartbeat from client for 30 sec - exiting 15:13:57 (5832): timer handler: client dead, exiting 15:13:58 (5832): No heartbeat from client for 30 sec - exiting 15:13:58 (5832): timer handler: client dead, exiting 15:13:59 (5832): No heartbeat from client for 30 sec - exiting 15:13:59 (5832): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3116, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4312, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... G09:54:40 (3044): No heartbeat from client for 30 sec - exiting 09:54:40 (3044): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4104, selfPID=4104, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3580, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:58:34 (5836): No heartbeat from client for 30 sec - exiting 17:58:34 (5836): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5380, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4744, selfPID=4080, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3416, selfPID=2924, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5224, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5688, selfPID=2680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5000, selfPID=3632, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4032, selfPID=3812, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2908, selfPID=3304, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5100, iMonCtr=2 Model crash detected, will try to restart... 20:14:29 (3700): No heartbeat from client for 30 sec - exiting 20:14:29 (3700): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3748, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:21:35 (4696): No heartbeat from client for 30 sec - exiting 18:21:35 (4696): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:27 (4320): No heartbeat from client for 30 sec - exiting 20:06:27 (4320): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:04:26 (4544): No heartbeat from client for 30 sec - exiting 14:04:26 (4544): timer handler: client dead, exiting 14:04:27 (4544): No heartbeat from client for 30 sec - exiting 14:04:27 (4544): timer handler: client dead, exiting 14:04:28 (4544): No heartbeat from client for 30 sec - exiting 14:04:28 (4544): timer handler: client dead, exiting 14:04:30 (4544): No heartbeat from client for 30 sec - exiting 14:04:30 (4544): timer handler: client dead, exiting 14:04:31 (4544): No heartbeat from client for 30 sec - exiting 14:04:31 (4544): timer handler: client dead, exiting 14:04:32 (4544): No heartbeat from client for 30 sec - exiting 14:04:32 (4544): timer handler: client dead, exiting 14:04:33 (4544): No heartbeat from client for 30 sec - exiting 14:04:33 (4544): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:36 (3412): No heartbeat from client for 30 sec - exiting 18:05:36 (3412): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5816, selfPID=5816, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CGlobal Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5364, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 18:00:23 (364): No heartbeat from client for 30 sec - exiting 18:00:23 (364): timer handler: client dead, exiting 18:00:24 (364): No heartbeat from client for 30 sec - exiting 18:00:24 (364): timer handler: client dead, exiting 18:00:25 (364): No heartbeat from client for 30 sec - exiting 18:00:25 (364): timer handler: client dead, exiting 18:00:26 (364): No heartbeat from client for 30 sec - exiting 18:00:26 (364): timer handler: client dead, exiting 18:00:27 (364): No heartbeat from client for 30 sec - exiting 18:00:27 (364): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:13:54 (3816): No heartbeat from client for 30 sec - exiting 18:13:54 (3816): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5156, selfPID=5156, iMonCtr=2 GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5384, selfPID=5192, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:52:54 (4628): No heartbeat from client for 30 sec - exiting 17:52:55 (4628): timer handler: client dead, exiting 17:52:56 (4628): No heartbeat from client for 30 sec - exiting 17:52:56 (4628): timer handler: client dead, exiting 17:52:57 (4628): No heartbeat from client for 30 sec - exiting 17:52:57 (4628): timer handler: client dead, exiting 17:52:58 (4628): No heartbeat from client for 30 sec - exiting 17:52:58 (4628): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:16:17 (4540): No heartbeat from client for 30 sec - exiting 14:16:17 (4540): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:11 (1048): No heartbeat from client for 30 sec - exiting 17:47:11 (1048): timer handler: client dead, exiting 17:47:12 (1048): No heartbeat from client for 30 sec - exiting 17:47:12 (1048): timer handler: client dead, exiting 17:47:13 (1048): No heartbeat from client for 30 sec - exiting 17:47:13 (1048): timer handler: client dead, exiting 17:47:14 (1048): No heartbeat from client for 30 sec - exiting 17:47:14 (1048): timer handler: client dead, exiting 17:47:15 (1048): No heartbeat from client for 30 sec - exiting 17:47:15 (1048): timer handler: client dead, exiting 17:47:17 (1048): No heartbeat from client for 30 sec - exiting 17:47:17 (1048): timer handler: client dead, exiting 17:47:18 (1048): No heartbeat from client for 30 sec - exiting 17:47:18 (1048): timer handler: client dead, exiting 17:47:19 (1048): No heartbeat from client for 30 sec - exiting 17:47:19 (1048): timer handler: client dead, exiting 17:47:20 (1048): No heartbeat from client for 30 sec - exiting 17:47:20 (1048): timer handler: client dead, exiting 17:47:21 (1048): No heartbeat from client for 30 sec - exiting 17:47:21 (1048): timer handler: client dead, exiting 17:47:22 (1048): No heartbeat from client for 30 sec - exiting 17:47:22 (1048): timer handler: client dead, exiting 17:47:23 (1048): No heartbeat from client for 30 sec - exiting 17:47:23 (1048): timer handler: client dead, exiting 17:47:24 (1048): No heartbeat from client for 30 sec - exiting 17:47:24 (1048): timer handler: client dead, exiting 17:47:25 (1048): No heartbeat from client for 30 sec - exiting 17:47:25 (1048): timer handler: client dead, exiting 17:47:26 (1048): No heartbeat from client for 30 sec - exiting 17:47:26 (1048): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Gl20:45:07 (5592): No heartbeat from client for 30 sec - exiting 20:45:07 (5592): timer handler: client dead, exiting 20:45:08 (5592): No heartbeat from client for 30 sec - exiting 20:45:08 (5592): timer handler: client dead, exiting 20:45:09 (5592): No heartbeat from client for 30 sec - exiting 20:45:09 (5592): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:10 (5592): No heartbeat from client for 30 sec - exiting 20:45:10 (5592): timer handler: client dead, exiting 09:24:19 (1564): No heartbeat from client for 30 sec - exiting 09:24:20 (1564): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:22:44 (4908): No heartbeat from client for 30 sec - exiting 09:22:44 (4908): timer handler: client dead, exiting 09:22:45 (4908): No heartbeat from client for 30 sec - exiting 09:22:45 (4908): timer handler: client dead, exiting 09:22:46 (4908): No heartbeat from client for 30 sec - exiting 09:22:46 (4908): timer handler: client dead, exiting 09:22:47 (4908): No heartbeat from client for 30 sec - exiting 09:22:47 (4908): timer handler: client dead, exiting 09:22:48 (4908): No heartbeat from client for 30 sec - exiting 09:22:48 (4908): timer handler: client dead, exiting 09:22:49 (4908): No heartbeat from client for 30 sec - exiting 09:22:49 (4908): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3048, selfPID=3220, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5624, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4816, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3472, selfPID=3036, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadam3p_pnw_hcw5_2011_1_009287467/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1632, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 19:47:06 (1632): called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_hcw5_2011_1_009287467_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Feb 2015 11:45:47	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	127,019	354,883	2.7939
24 Feb 2015 18:06:32	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	115,499	323,356	2.7996
18 Feb 2015 17:19:23	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	103,979	291,223	2.8008
29 Jan 2015 20:14:00	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	92,459	259,733	2.8092
20 Jan 2015 17:53:40	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	80,939	227,811	2.8146
12 Jan 2015 13:33:58	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	69,419	196,081	2.8246
02 Jan 2015 17:52:23	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	57,899	163,581	2.8253
29 Dec 2014 19:35:47	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	46,379	130,188	2.8070
25 Dec 2014 19:49:51	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	34,859	97,257	2.7900
18 Dec 2014 17:56:12	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	23,339	64,638	2.7695
11 Dec 2014 17:22:01	1272486	17549778	hadam3p_pnw_hcw5_2011_1_009287467_0	11,819	33,698	2.8512