Task 16420849

Name	hadam3p_anz_nbo0_2012_1_008602736_1
Workunit	8749248
Created	26 Mar 2014, 22:36:06 UTC
Sent	26 Mar 2014, 22:43:12 UTC
Report deadline	9 Mar 2015, 4:03:12 UTC
Received	14 Apr 2014, 9:47:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1302497
Run time	9 days 9 hours 7 min 59 sec
CPU time	8 days 16 hours 5 min 14 sec
Validate state	Invalid
Credit	2,993.82
Device peak FLOPS	1.71 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.28</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:39:36 (5904): No heartbeat from core client for 30 sec - exiting 08:39:37 (5904): No heartbeat from core client for 30 sec - exiting 08:39:38 (5904): No heartbeat from core client for 30 sec - exiting 08:39:39 (5904): No heartbeat from core client for 30 sec - exiting 08:39:40 (5904): No heartbeat from core client for 30 sec - exiting 08:39:41 (5904): No heartbeat from core client for 30 sec - exiting 08:39:42 (5904): No heartbeat from core client for 30 sec - exiting 08:39:43 (5904): No heartbeat from core client for 30 sec - exiting 08:39:44 (5904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12816, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13080, selfPID=5432, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:00:14 (13320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15392, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14232, selfPID=6384, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8748, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4588, selfPID=7972, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6952, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5876, selfPID=8948, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPIDC0, selontroller:: CPDN proce ss is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16340, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 09:14:26 (6412): No heartbeat from core client for 30 sec - exiting 09:14:28 (6412): No heartbeat from core client for 30 sec - exiting 09:14:29 (6412): No heartbeat from core client for 30 sec - exiting 09:14:30 (6412): No heartbeat from core client for 30 sec - exiting 09:14:31 (6412): No heartbeat from core client for 30 sec - exiting 09:14:32 (6412): No heartbeat from core client for 30 sec - exiting 09:14:33 (6412): No heartbeat from core client for 30 sec - exiting 09:14:34 (6412): No heartbeat from core client for 30 sec - exiting 09:14:35 (6412): No heartbeat from core client for 30 sec - exiting 09:14:36 (6412): No heartbeat from core client for 30 sec - exiting 09:14:37 (6412): No heartbeat from core client for 30 sec - exiting 09:14:38 (6412): No heartbeat from core client for 30 sec - exiting 09:14:39 (6412): No heartbeat from core client for 30 sec - exiting 09:14:40 (6412): No heartbeat from core client for 30 sec - exiting 09:14:41 (6412): No heartbeat from core client for 30 sec - exiting 09:14:42 (6412): No heartbeat from core client for 30 sec - exiting 09:14:43 (6412): No heartbeat from core client for 30 sec - exiting 09:14:44 (6412): No heartbeat from core client for 30 sec - exiting 09:14:45 (6412): No heartbeat from core client for 30 sec - exiting 09:14:46 (6412): No heartbeat from core client for 30 sec - exiting 09:14:47 (6412): No heartbeat from core client for 30 sec - exiting 09:14:48 (6412): No heartbeat from core client for 30 sec - exiting 09:14:49 (6412): No heartbeat from core client for 30 sec - exiting 09:14:50 (6412): No heartbeat from core client for 30 sec - exiting 09:14:51 (6412): No heartbeat from core client for 30 sec - exiting 09:14:52 (6412): No heartbeat from core client for 30 sec - exiting 09:14:53 (6412): No heartbeat from core client for 30 sec - exiting 09:14:54 (6412): No heartbeat from core client for 30 sec - exiting 09:14:55 (6412): No heartbeat from core client for 30 sec - exiting 09:14:56 (6412): No heartbeat from core client for 30 sec - exiting 09:14:57 (6412): No heartbeat from core client for 30 sec - exiting 09:14:58 (6412): No heartbeat from core client for 30 sec - exiting 09:14:59 (6412): No heartbeat from core client for 30 sec - exiting 09:15:00 (6412): No heartbeat from core client for 30 sec - exiting 09:15:01 (6412): No heartbeat from core client for 30 sec - exiting 09:15:02 (6412): No heartbeat from core client for 30 sec - exiting 09:15:03 (6412): No heartbeat from core client for 30 sec - exiting 09:15:04 (6412): No heartbeat from core client for 30 sec - exiting 09:15:05 (6412): No heartbeat from core client for 30 sec - exiting 09:15:06 (6412): No heartbeat from core client for 30 sec - exiting 09:15:07 (6412): No heartbeat from core client for 30 sec - exiting 09:15:08 (6412): No heartbeat from core client for 30 sec - exiting 09:15:09 (6412): No heartbeat from core client for 30 sec - exiting 09:15:10 (6412): No heartbeat from core client for 30 sec - exiting 09:15:11 (6412): No heartbeat from core client for 30 sec - exiting 09:15:12 (6412): No heartbeat from core client for 30 sec - exiting 09:15:13 (6412): No heartbeat from core client for 30 sec - exiting 09:15:14 (6412): No heartbeat from core client for 30 sec - exiting 09:15:15 (6412): No heartbeat from core client for 30 sec - exiting 09:15:16 (6412): No heartbeat from core client for 30 sec - exiting 09:15:17 (6412): No heartbeat from core client for 30 sec - exiting 09:15:18 (6412): No heartbeat from core client for 30 sec - exiting 09:15:19 (6412): No heartbeat from core client for 30 sec - exiting 09:15:20 (6412): No heartbeat from core client for 30 sec - exiting 09:15:21 (6412): No heartbeat from core client for 30 sec - exiting 09:15:22 (6412): No heartbeat from core client for 30 sec - exiting 09:15:23 (6412): No heartbeat from core client for 30 sec - exiting 09:15:24 (6412): No heartbeat from core client for 30 sec - exiting 09:15:25 (6412): No heartbeat from core client for 30 sec - exiting 09:15:26 (6412): No heartbeat from core client for 30 sec - exiting 09:15:27 (6412): No heartbeat from core client for 30 sec - exiting 09:15:28 (6412): No heartbeat from core client for 30 sec - exiting 09:15:29 (6412): No heartbeat from core client for 30 sec - exiting 09:15:30 (6412): No heartbeat from core client for 30 sec - exiting 09:15:31 (6412): No heartbeat from core client for 30 sec - exiting 09:15:32 (6412): No heartbeat from core client for 30 sec - exiting 09:15:33 (6412): No heartbeat from core client for 30 sec - exiting 09:15:34 (6412): No heartbeat from core client for 30 sec - exiting 09:15:35 (6412): No heartbeat from core client for 30 sec - exiting 09:15:36 (6412): No heartbeat from core client for 30 sec - exiting 09:15:37 (6412): No heartbeat from core client for 30 sec - exiting 09:15:38 (6412): No heartbeat from core client for 30 sec - exiting 09:15:39 (6412): No heartbeat from core client for 30 sec - exiting 09:15:40 (6412): No heartbeat from core client for 30 sec - exiting 09:15:41 (6412): No heartbeat from core client for 30 sec - exiting 09:15:42 (6412): No heartbeat from core client for 30 sec - exiting 09:15:43 (6412): No heartbeat from core client for 30 sec - exiting 09:15:44 (6412): No heartbeat from core client for 30 sec - exiting 09:15:45 (6412): No heartbeat from core client for 30 sec - exiting 09:15:46 (6412): No heartbeat from core client for 30 sec - exiting 09:15:47 (6412): No heartbeat from core client for 30 sec - exiting 09:15:48 (6412): No heartbeat from core client for 30 sec - exiting 09:15:49 (6412): No heartbeat from core client for 30 sec - exiting 09:15:50 (6412): No heartbeat from core client for 30 sec - exiting 09:15:51 (6412): No heartbeat from core client for 30 sec - exiting 09:15:52 (6412): No heartbeat from core client for 30 sec - exiting 09:15:53 (6412): No heartbeat from core client for 30 sec - exiting 09:15:54 (6412): No heartbeat from core client for 30 sec - exiting 09:15:55 (6412): No heartbeat from core client for 30 sec - exiting 09:15:56 (6412): No heartbeat from core client for 30 sec - exiting 09:15:57 (6412): No heartbeat from core client for 30 sec - exiting 09:15:58 (6412): No heartbeat from core client for 30 sec - exiting 09:15:59 (6412): No heartbeat from core client for 30 sec - exiting 09:16:00 (6412): No heartbeat from core client for 30 sec - exiting 09:16:01 (6412): No heartbeat from core client for 30 sec - exiting 09:16:02 (6412): No heartbeat from core client for 30 sec - exiting 09:16:03 (6412): No heartbeat from core client for 30 sec - exiting 09:16:04 (6412): No heartbeat from core client for 30 sec - exiting 09:16:05 (6412): No heartbeat from core client for 30 sec - exiting 09:16:06 (6412): No heartbeat from core client for 30 sec - exiting 09:16:07 (6412): No heartbeat from core client for 30 sec - exiting 09:16:08 (6412): No heartbeat from core client for 30 sec - exiting 09:16:09 (6412): No heartbeat from core client for 30 sec - exiting 09:16:10 (6412): No heartbeat from core client for 30 sec - exiting 09:16:11 (6412): No heartbeat from core client for 30 sec - exiting 09:16:12 (6412): No heartbeat from core client for 30 sec - exiting 09:16:13 (6412): No heartbeat from core client for 30 sec - exiting 09:16:14 (6412): No heartbeat from core client for 30 sec - exiting 09:16:15 (6412): No heartbeat from core client for 30 sec - exiting 09:16:16 (6412): No heartbeat from core client for 30 sec - exiting 09:16:17 (6412): No heartbeat from core client for 30 sec - exiting 09:16:18 (6412): No heartbeat from core client for 30 sec - exiting 09:16:19 (6412): No heartbeat from core client for 30 sec - exiting 09:16:20 (6412): No heartbeat from core client for 30 sec - exiting 09:16:21 (6412): No heartbeat from core client for 30 sec - exiting 09:16:22 (6412): No heartbeat from core client for 30 sec - exiting 09:16:23 (6412): No heartbeat from core client for 30 sec - exiting 09:16:24 (6412): No heartbeat from core client for 30 sec - exiting 09:16:25 (6412): No heartbeat from core client for 30 sec - exiting 09:16:26 (6412): No heartbeat from core client for 30 sec - exiting 09:16:27 (6412): No heartbeat from core client for 30 sec - exiting 09:16:28 (6412): No heartbeat from core client for 30 sec - exiting 09:16:29 (6412): No heartbeat from core client for 30 sec - exiting 09:16:30 (6412): No heartbeat from core client for 30 sec - exiting 09:16:31 (6412): No heartbeat from core client for 30 sec - exiting 09:16:32 (6412): No heartbeat from core client for 30 sec - exiting 09:16:33 (6412): No heartbeat from core client for 30 sec - exiting 09:16:34 (6412): No heartbeat from core client for 30 sec - exiting 09:16:35 (6412): No heartbeat from core client for 30 sec - exiting 09:16:36 (6412): No heartbeat from core client for 30 sec - exiting 09:16:37 (6412): No heartbeat from core client for 30 sec - exiting 09:16:38 (6412): No heartbeat from core client for 30 sec - exiting 09:16:39 (6412): No heartbeat from core client for 30 sec - exiting 09:16:40 (6412): No heartbeat from core client for 30 sec - exiting 09:16:41 (6412): No heartbeat from core client for 30 sec - exiting 09:16:42 (6412): No heartbeat from core client for 30 sec - exiting 09:16:43 (6412): No heartbeat from core client for 30 sec - exiting 09:16:44 (6412): No heartbeat from core client for 30 sec - exiting 09:16:45 (6412): No heartbeat from core client for 30 sec - exiting 09:16:46 (6412): No heartbeat from core client for 30 sec - exiting 09:16:47 (6412): No heartbeat from core client for 30 sec - exiting 09:16:48 (6412): No heartbeat from core client for 30 sec - exiting 09:16:49 (6412): No heartbeat from core client for 30 sec - exiting 09:16:50 (6412): No heartbeat from core client for 30 sec - exiting 09:16:51 (6412): No heartbeat from core client for 30 sec - exiting 09:16:52 (6412): No heartbeat from core client for 30 sec - exiting 09:16:53 (6412): No heartbeat from core client for 30 sec - exiting 09:16:54 (6412): No heartbeat from core client for 30 sec - exiting 09:16:55 (6412): No heartbeat from core client for 30 sec - exiting 09:16:56 (6412): No heartbeat from core client for 30 sec - exiting 09:16:57 (6412): No heartbeat from core client for 30 sec - exiting 09:16:58 (6412): No heartbeat from core client for 30 sec - exiting 09:16:59 (6412): No heartbeat from core client for 30 sec - exiting 09:17:00 (6412): No heartbeat from core client for 30 sec - exiting 09:17:01 (6412): No heartbeat from core client for 30 sec - exiting 09:17:02 (6412): No heartbeat from core client for 30 sec - exiting 09:17:03 (6412): No heartbeat from core client for 30 sec - exiting 09:17:04 (6412): No heartbeat from core client for 30 sec - exiting 09:17:05 (6412): No heartbeat from core client for 30 sec - exiting 09:17:06 (6412): No heartbeat from core client for 30 sec - exiting 09:17:07 (6412): No heartbeat from core client for 30 sec - exiting 09:17:08 (6412): No heartbeat from core client for 30 sec - exiting 09:17:09 (6412): No heartbeat from core client for 30 sec - exiting 09:17:10 (6412): No heartbeat from core client for 30 sec - exiting 09:17:11 (6412): No heartbeat from core client for 30 sec - exiting 09:17:12 (6412): No heartbeat from core client for 30 sec - exiting 09:17:13 (6412): No heartbeat from core client for 30 sec - exiting 09:17:14 (6412): No heartbeat from core client for 30 sec - exiting 09:17:15 (6412): No heartbeat from core client for 30 sec - exiting 09:17:16 (6412): No heartbeat from core client for 30 sec - exiting 09:17:17 (6412): No heartbeat from core client for 30 sec - exiting 09:17:18 (6412): No heartbeat from core client for 30 sec - exiting 09:17:19 (6412): No heartbeat from core client for 30 sec - exiting 09:17:20 (6412): No heartbeat from core client for 30 sec - exiting 09:17:21 (6412): No heartbeat from core client for 30 sec - exiting 09:17:22 (6412): No heartbeat from core client for 30 sec - exiting 09:17:23 (6412): No heartbeat from core client for 30 sec - exiting 09:17:24 (6412): No heartbeat from core client for 30 sec - exiting 09:17:25 (6412): No heartbeat from core client for 30 sec - exiting 09:17:26 (6412): No heartbeat from core client for 30 sec - exiting 09:17:27 (6412): No heartbeat from core client for 30 sec - exiting 09:17:28 (6412): No heartbeat from core client for 30 sec - exiting 09:17:29 (6412): No heartbeat from core client for 30 sec - exiting 09:17:30 (6412): No heartbeat from core client for 30 sec - exiting 09:17:31 (6412): No heartbeat from core client for 30 sec - exiting 09:17:32 (6412): No heartbeat from core client for 30 sec - exiting 09:17:33 (6412): No heartbeat from core client for 30 sec - exiting 09:17:34 (6412): No heartbeat from core client for 30 sec - exiting 09:17:35 (6412): No heartbeat from core client for 30 sec - exiting 09:17:36 (6412): No heartbeat from core client for 30 sec - exiting 09:17:37 (6412): No heartbeat from core client for 30 sec - exiting 09:17:38 (6412): No heartbeat from core client for 30 sec - exiting 09:17:39 (6412): No heartbeat from core client for 30 sec - exiting 09:17:40 (6412): No heartbeat from core client for 30 sec - exiting 09:17:41 (6412): No heartbeat from core client for 30 sec - exiting 09:17:42 (6412): No heartbeat from core client for 30 sec - exiting 09:17:43 (6412): No heartbeat from core client for 30 sec - exiting 09:17:44 (6412): No heartbeat from core client for 30 sec - exiting 09:17:45 (6412): No heartbeat from core client for 30 sec - exiting 09:17:46 (6412): No heartbeat from core client for 30 sec - exiting 09:17:47 (6412): No heartbeat from core client for 30 sec - exiting 09:17:48 (6412): No heartbeat from core client for 30 sec - exiting 09:17:49 (6412): No heartbeat from core client for 30 sec - exiting 09:17:50 (6412): No heartbeat from core client for 30 sec - exiting 09:17:51 (6412): No heartbeat from core client for 30 sec - exiting 09:17:52 (6412): No heartbeat from core client for 30 sec - exiting 09:17:53 (6412): No heartbeat from core client for 30 sec - exiting 09:17:54 (6412): No heartbeat from core client for 30 sec - exiting 09:17:55 (6412): No heartbeat from core client for 30 sec - exiting 09:17:56 (6412): No heartbeat from core client for 30 sec - exiting 09:17:57 (6412): No heartbeat from core client for 30 sec - exiting 09:17:58 (6412): No heartbeat from core client for 30 sec - exiting 09:17:59 (6412): No heartbeat from core client for 30 sec - exiting 09:18:00 (6412): No heartbeat from core client for 30 sec - exiting 09:18:01 (6412): No heartbeat from core client for 30 sec - exiting 09:18:02 (6412): No heartbeat from core client for 30 sec - exiting 09:18:03 (6412): No heartbeat from core client for 30 sec - exiting 09:18:04 (6412): No heartbeat from core client for 30 sec - exiting 09:18:05 (6412): No heartbeat from core client for 30 sec - exiting 09:18:06 (6412): No heartbeat from core client for 30 sec - exiting 09:18:07 (6412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8684, selfPID=8684, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14772, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11712, selfPID=10956, iMonCtr=1 Model crash detected, will try to restart... 21:45:19 (6060): No heartbeat from core client for 30 sec - exiting 21:45:20 (6060): No heartbeat from core client for 30 sec - exiting 21:45:21 (6060): No heartbeat from core client for 30 sec - exiting 21:45:22 (6060): No heartbeat from core client for 30 sec - exiting 21:45:23 (6060): No heartbeat from core client for 30 sec - exiting 21:45:24 (6060): No heartbeat from core client for 30 sec - exiting 21:45:25 (6060): No heartbeat from core client for 30 sec - exiting 21:45:26 (6060): No heartbeat from core client for 30 sec - exiting 21:45:27 (6060): No heartbeat from core client for 30 sec - exiting 21:45:28 (6060): No heartbeat from core client for 30 sec - exiting 21:45:29 (6060): No heartbeat from core client for 30 sec - exiting 21:45:30 (6060): No heartbeat from core client for 30 sec - exiting 21:45:31 (6060): No heartbeat from core client for 30 sec - exiting 21:45:32 (6060): No heartbeat from core client for 30 sec - exiting 21:45:33 (6060): No heartbeat from core client for 30 sec - exiting 21:45:34 (6060): No heartbeat from core client for 30 sec - exiting 21:45:35 (6060): No heartbeat from core client for 30 sec - exiting 21:45:36 (6060): No heartbeat from core client for 30 sec - exiting 21:45:37 (6060): No heartbeat from core client for 30 sec - exiting 21:45:38 (6060): No heartbeat from core client for 30 sec - exiting 21:45:39 (6060): No heartbeat from core client for 30 sec - exiting 21:45:40 (6060): No heartbeat from core client for 30 sec - exiting 21:45:41 (6060): No heartbeat from core client for 30 sec - exiting 21:45:42 (6060): No heartbeat from core client for 30 sec - exiting 21:45:43 (6060): No heartbeat from core client for 30 sec - exiting 21:45:44 (6060): No heartbeat from core client for 30 sec - exiting 21:45:45 (6060): No heartbeat from core client for 30 sec - exiting 21:45:46 (6060): No heartbeat from core client for 30 sec - exiting 21:45:47 (6060): No heartbeat from core client for 30 sec - exiting 21:45:48 (6060): No heartbeat from core client for 30 sec - exiting 21:45:49 (6060): No heartbeat from core client for 30 sec - exiting 21:45:50 (6060): No heartbeat from core client for 30 sec - exiting 21:45:51 (6060): No heartbeat from core client for 30 sec - exiting 21:45:52 (6060): No heartbeat from core client for 30 sec - exiting 21:45:53 (6060): No heartbeat from core client for 30 sec - exiting 21:45:54 (6060): No heartbeat from core client for 30 sec - exiting 21:45:55 (6060): No heartbeat from core client for 30 sec - exiting 21:45:56 (6060): No heartbeat from core client for 30 sec - exiting 21:45:57 (6060): No heartbeat from core client for 30 sec - exiting 21:45:58 (6060): No heartbeat from core client for 30 sec - exiting 21:45:59 (6060): No heartbeat from core client for 30 sec - exiting 21:46:00 (6060): No heartbeat from core client for 30 sec - exiting 21:46:01 (6060): No heartbeat from core client for 30 sec - exiting 21:46:02 (6060): No heartbeat from core client for 30 sec - exiting 21:46:03 (6060): No heartbeat from core client for 30 sec - exiting 21:46:04 (6060): No heartbeat from core client for 30 sec - exiting 21:46:05 (6060): No heartbeat from core client for 30 sec - exiting 21:46:06 (6060): No heartbeat from core client for 30 sec - exiting 21:46:07 (6060): No heartbeat from core client for 30 sec - exiting 21:46:08 (6060): No heartbeat from core client for 30 sec - exiting 21:46:09 (6060): No heartbeat from core client for 30 sec - exiting 21:46:10 (6060): No heartbeat from core client for 30 sec - exiting 21:46:11 (6060): No heartbeat from core client for 30 sec - exiting 21:46:12 (6060): No heartbeat from core client for 30 sec - exiting 21:46:13 (6060): No heartbeat from core client for 30 sec - exiting 21:46:14 (6060): No heartbeat from core client for 30 sec - exiting 21:46:15 (6060): No heartbeat from core client for 30 sec - exiting 21:46:16 (6060): No heartbeat from core client for 30 sec - exiting 21:46:17 (6060): No heartbeat from core client for 30 sec - exiting 21:46:18 (6060): No heartbeat from core client for 30 sec - exiting 21:46:19 (6060): No heartbeat from core client for 30 sec - exiting 21:46:20 (6060): No heartbeat from core client for 30 sec - exiting 21:46:21 (6060): No heartbeat from core client for 30 sec - exiting 21:46:22 (6060): No heartbeat from core client for 30 sec - exiting 21:46:23 (6060): No heartbeat from core client for 30 sec - exiting 21:46:24 (6060): No heartbeat from core client for 30 sec - exiting 21:46:25 (6060): No heartbeat from core client for 30 sec - exiting 21:46:26 (6060): No heartbeat from core client for 30 sec - exiting 21:46:27 (6060): No heartbeat from core client for 30 sec - exiting 21:46:28 (6060): No heartbeat from core client for 30 sec - exiting 21:46:29 (6060): No heartbeat from core client for 30 sec - exiting 21:46:30 (6060): No heartbeat from core client for 30 sec - exiting 21:46:31 (6060): No heartbeat from core client for 30 sec - exiting 21:46:32 (6060): No heartbeat from core client for 30 sec - exiting 21:46:33 (6060): No heartbeat from core client for 30 sec - exiting 21:46:34 (6060): No heartbeat from core client for 30 sec - exiting 21:46:35 (6060): No heartbeat from core client for 30 sec - exiting 21:46:36 (6060): No heartbeat from core client for 30 sec - exiting 21:46:37 (6060): No heartbeat from core client for 30 sec - exiting 21:46:38 (6060): No heartbeat from core client for 30 sec - exiting 21:46:39 (6060): No heartbeat from core client for 30 sec - exiting 21:46:40 (6060): No heartbeat from core client for 30 sec - exiting 21:46:41 (6060): No heartbeat from core client for 30 sec - exiting 21:46:42 (6060): No heartbeat from core client for 30 sec - exiting 21:46:43 (6060): No heartbeat from core client for 30 sec - exiting 21:46:44 (6060): No heartbeat from core client for 30 sec - exiting 21:46:45 (6060): No heartbeat from core client for 30 sec - exiting 21:46:46 (6060): No heartbeat from core client for 30 sec - exiting 21:46:47 (6060): No heartbeat from core client for 30 sec - exiting 21:46:48 (6060): No heartbeat from core client for 30 sec - exiting 21:46:49 (6060): No heartbeat from core client for 30 sec - exiting 21:46:50 (6060): No heartbeat from core client for 30 sec - exiting 21:46:51 (6060): No heartbeat from core client for 30 sec - exiting 21:46:52 (6060): No heartbeat from core client for 30 sec - exiting 21:46:53 (6060): No heartbeat from core client for 30 sec - exiting 21:46:54 (6060): No heartbeat from core client for 30 sec - exiting 21:46:55 (6060): No heartbeat from core client for 30 sec - exiting 21:46:56 (6060): No heartbeat from core client for 30 sec - exiting 21:46:57 (6060): No heartbeat from core client for 30 sec - exiting 21:46:58 (6060): No heartbeat from core client for 30 sec - exiting 21:46:59 (6060): No heartbeat from core client for 30 sec - exiting 21:47:00 (6060): No heartbeat from core client for 30 sec - exiting 21:47:01 (6060): No heartbeat from core client for 30 sec - exiting 21:47:02 (6060): No heartbeat from core client for 30 sec - exiting 21:47:03 (6060): No heartbeat from core client for 30 sec - exiting 21:47:04 (6060): No heartbeat from core client for 30 sec - exiting 21:47:05 (6060): No heartbeat from core client for 30 sec - exiting 21:47:06 (6060): No heartbeat from core client for 30 sec - exiting 21:47:07 (6060): No heartbeat from core client for 30 sec - exiting 21:47:08 (6060): No heartbeat from core client for 30 sec - exiting 21:47:09 (6060): No heartbeat from core client for 30 sec - exiting 21:47:10 (6060): No heartbeat from core client for 30 sec - exiting 21:47:11 (6060): No heartbeat from core client for 30 sec - exiting 21:47:12 (6060): No heartbeat from core client for 30 sec - exiting 21:47:13 (6060): No heartbeat from core client for 30 sec - exiting 21:47:14 (6060): No heartbeat from core client for 30 sec - exiting 21:47:15 (6060): No heartbeat from core client for 30 sec - exiting 21:47:16 (6060): No heartbeat from core client for 30 sec - exiting 21:47:17 (6060): No heartbeat from core client for 30 sec - exiting 21:47:18 (6060): No heartbeat from core client for 30 sec - exiting 21:47:19 (6060): No heartbeat from core client for 30 sec - exiting 21:47:20 (6060): No heartbeat from core client for 30 sec - exiting 21:47:21 (6060): No heartbeat from core client for 30 sec - exiting 21:47:22 (6060): No heartbeat from core client for 30 sec - exiting 21:47:23 (6060): No heartbeat from core client for 30 sec - exiting 21:47:24 (6060): No heartbeat from core client for 30 sec - exiting 21:47:25 (6060): No heartbeat from core client for 30 sec - exiting 21:47:26 (6060): No heartbeat from core client for 30 sec - exiting 21:47:27 (6060): No heartbeat from core client for 30 sec - exiting 21:47:28 (6060): No heartbeat from core client for 30 sec - exiting 21:47:29 (6060): No heartbeat from core client for 30 sec - exiting 21:47:30 (6060): No heartbeat from core client for 30 sec - exiting 21:47:31 (6060): No heartbeat from core client for 30 sec - exiting 21:47:32 (6060): No heartbeat from core client for 30 sec - exiting 21:47:33 (6060): No heartbeat from core client for 30 sec - exiting 21:47:34 (6060): No heartbeat from core client for 30 sec - exiting 21:47:35 (6060): No heartbeat from core client for 30 sec - exiting 21:47:36 (6060): No heartbeat from core client for 30 sec - exiting 21:47:37 (6060): No heartbeat from core client for 30 sec - exiting 21:47:38 (6060): No heartbeat from core client for 30 sec - exiting 21:47:39 (6060): No heartbeat from core client for 30 sec - exiting 21:47:40 (6060): No heartbeat from core client for 30 sec - exiting 21:47:41 (6060): No heartbeat from core client for 30 sec - exiting 21:47:42 (6060): No heartbeat from core client for 30 sec - exiting 21:47:43 (6060): No heartbeat from core client for 30 sec - exiting 21:47:44 (6060): No heartbeat from core client for 30 sec - exiting 21:47:45 (6060): No heartbeat from core client for 30 sec - exiting 21:47:46 (6060): No heartbeat from core client for 30 sec - exiting 21:47:47 (6060): No heartbeat from core client for 30 sec - exiting 21:47:48 (6060): No heartbeat from core client for 30 sec - exiting 21:47:49 (6060): No heartbeat from core client for 30 sec - exiting 21:47:50 (6060): No heartbeat from core client for 30 sec - exiting 21:47:51 (6060): No heartbeat from core client for 30 sec - exiting 21:47:52 (6060): No heartbeat from core client for 30 sec - exiting 21:47:53 (6060): No heartbeat from core client for 30 sec - exiting 21:47:54 (6060): No heartbeat from core client for 30 sec - exiting 21:47:55 (6060): No heartbeat from core client for 30 sec - exiting 21:47:56 (6060): No heartbeat from core client for 30 sec - exiting 21:47:57 (6060): No heartbeat from core client for 30 sec - exiting 21:47:58 (6060): No heartbeat from core client for 30 sec - exiting 21:47:59 (6060): No heartbeat from core client for 30 sec - exiting 21:48:00 (6060): No heartbeat from core client for 30 sec - exiting 21:48:01 (6060): No heartbeat from core client for 30 sec - exiting 21:48:02 (6060): No heartbeat from core client for 30 sec - exiting 21:48:03 (6060): No heartbeat from core client for 30 sec - exiting 21:48:04 (6060): No heartbeat from core client for 30 sec - exiting 21:48:05 (6060): No heartbeat from core client for 30 sec - exiting 21:48:06 (6060): No heartbeat from core client for 30 sec - exiting 21:48:07 (6060): No heartbeat from core client for 30 sec - exiting 21:48:08 (6060): No heartbeat from core client for 30 sec - exiting 21:48:09 (6060): No heartbeat from core client for 30 sec - exiting 21:48:10 (6060): No heartbeat from core client for 30 sec - exiting 21:48:11 (6060): No heartbeat from core client for 30 sec - exiting 21:48:12 (6060): No heartbeat from core client for 30 sec - exiting 21:48:13 (6060): No heartbeat from core client for 30 sec - exiting 21:48:14 (6060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14600, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7608, selfPID=2320, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_nbo0_2012_1_008602736_1_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nbo0_2012_1_008602736_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nbo0_2012_1_008602736_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nbo0_2012_1_008602736_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nbo0_2012_1_008602736_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_nbo0_2012_1_008602736_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Apr 2014 11:26:23	1302497	16420849	hadam3p_anz_nbo0_2012_1_008602736_1	69,419	667,035	9.6088
08 Apr 2014 22:02:26	1302497	16420849	hadam3p_anz_nbo0_2012_1_008602736_1	57,899	559,152	9.6574
06 Apr 2014 05:43:40	1302497	16420849	hadam3p_anz_nbo0_2012_1_008602736_1	46,379	452,641	9.7596
03 Apr 2014 08:10:39	1302497	16420849	hadam3p_anz_nbo0_2012_1_008602736_1	34,859	342,726	9.8318
31 Mar 2014 06:51:56	1302497	16420849	hadam3p_anz_nbo0_2012_1_008602736_1	23,339	227,836	9.7620
29 Mar 2014 01:54:44	1302497	16420849	hadam3p_anz_nbo0_2012_1_008602736_1	11,819	114,145	9.6578