Task 15221675

Name	hadam3p_pnw_z7s9_1968_1_006940609_1
Workunit	7143925
Created	1 Sep 2012, 11:53:50 UTC
Sent	10 Sep 2012, 8:08:04 UTC
Report deadline	23 Aug 2013, 13:28:04 UTC
Received	21 Oct 2012, 7:43:36 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1181641
Run time	6 days 10 hours 27 min 44 sec
CPU time	6 days 6 hours 4 min 58 sec
Validate state	Initial
Credit	3,005.88
Device peak FLOPS	2.25 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4476, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5608, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4732, selfPID=4852, iMonCtr=1 Model crash detected, will try to restart... 17:47:07 (4824): No heartbeat from core client for 30 sec - exiting 17:47:09 (4824): No heartbeat from core client for 30 sec - exiting 17:47:10 (4824): No heartbeat from core client for 30 sec - exiting 17:47:11 (4824): No heartbeat from core client for 30 sec - exiting 17:47:12 (4824): No heartbeat from core client for 30 sec - exiting 17:47:13 (4824): No heartbeat from core client for 30 sec - exiting 17:47:14 (4824): No heartbeat from core client for 30 sec - exiting 17:47:15 (4824): No heartbeat from core client for 30 sec - exiting 17:47:16 (4824): No heartbeat from core client for 30 sec - exiting 17:47:17 (4824): No heartbeat from core client for 30 sec - exiting 17:47:18 (4824): No heartbeat from core client for 30 sec - exiting 17:47:20 (4824): No heartbeat from core client for 30 sec - exiting 17:47:21 (4824): No heartbeat from core client for 30 sec - exiting 17:47:22 (4824): No heartbeat from core client for 30 sec - exiting 17:47:23 (4824): No heartbeat from core client for 30 sec - exiting 17:47:24 (4824): No heartbeat from core client for 30 sec - exiting 17:47:25 (4824): No heartbeat from core client for 30 sec - exiting 17:47:26 (4824): No heartbeat from core client for 30 sec - exiting 17:47:27 (4824): No heartbeat from core client for 30 sec - exiting 17:47:28 (4824): No heartbeat from core client for 30 sec - exiting 17:47:29 (4824): No heartbeat from core client for 30 sec - exiting 17:47:30 (4824): No heartbeat from core client for 30 sec - exiting 17:47:32 (4824): No heartbeat from core client for 30 sec - exiting 17:47:33 (4824): No heartbeat from core client for 30 sec - exiting 17:47:34 (4824): No heartbeat from core client for 30 sec - exiting 17:47:35 (4824): No heartbeat from core client for 30 sec - exiting 17:47:36 (4824): No heartbeat from core client for 30 sec - exiting 17:47:37 (4824): No heartbeat from core client for 30 sec - exiting 17:47:38 (4824): No heartbeat from core client for 30 sec - exiting 17:47:39 (4824): No heartbeat from core client for 30 sec - exiting 17:47:40 (4824): No heartbeat from core client for 30 sec - exiting 17:47:41 (4824): No heartbeat from core client for 30 sec - exiting 17:47:42 (4824): No heartbeat from core client for 30 sec - exiting 17:47:44 (4824): No heartbeat from core client for 30 sec - exiting 17:47:45 (4824): No heartbeat from core client for 30 sec - exiting 17:47:46 (4824): No heartbeat from core client for 30 sec - exiting 17:47:47 (4824): No heartbeat from core client for 30 sec - exiting 17:47:48 (4824): No heartbeat from core client for 30 sec - exiting 17:47:49 (4824): No heartbeat from core client for 30 sec - exiting 17:47:50 (4824): No heartbeat from core client for 30 sec - exiting 17:47:51 (4824): No heartbeat from core client for 30 sec - exiting 17:47:52 (4824): No heartbeat from core client for 30 sec - exiting 17:47:53 (4824): No heartbeat from core client for 30 sec - exiting 17:47:54 (4824): No heartbeat from core client for 30 sec - exiting 17:47:56 (4824): No heartbeat from core client for 30 sec - exiting 17:47:57 (4824): No heartbeat from core client for 30 sec - exiting 17:47:58 (4824): No heartbeat from core client for 30 sec - exiting 17:47:59 (4824): No heartbeat from core client for 30 sec - exiting 17:48:00 (4824): No heartbeat from core client for 30 sec - exiting 17:48:01 (4824): No heartbeat from core client for 30 sec - exiting 17:48:02 (4824): No heartbeat from core client for 30 sec - exiting 17:48:03 (4824): No heartbeat from core client for 30 sec - exiting 17:48:04 (4824): No heartbeat from core client for 30 sec - exiting 17:48:05 (4824): No heartbeat from core client for 30 sec - exiting 17:48:06 (4824): No heartbeat from core client for 30 sec - exiting 17:48:08 (4824): No heartbeat from core client for 30 sec - exiting 17:48:09 (4824): No heartbeat from core client for 30 sec - exiting 17:48:10 (4824): No heartbeat from core client for 30 sec - exiting 17:48:11 (4824): No heartbeat from core client for 30 sec - exiting 17:48:12 (4824): No heartbeat from core client for 30 sec - exiting 17:48:13 (4824): No heartbeat from core client for 30 sec - exiting 17:48:14 (4824): No heartbeat from core client for 30 sec - exiting 17:48:15 (4824): No heartbeat from core client for 30 sec - exiting 17:48:16 (4824): No heartbeat from core client for 30 sec - exiting 17:48:17 (4824): No heartbeat from core client for 30 sec - exiting 17:48:18 (4824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:54:13 (852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:14 (852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6560, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5564, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:11:48 (4496): No heartbeat from core client for 30 sec - exiting 16:11:49 (4496): No heartbeat from core client for 30 sec - exiting 16:11:50 (4496): No heartbeat from core client for 30 sec - exiting 16:11:51 (4496): No heartbeat from core client for 30 sec - exiting 16:11:52 (4496): No heartbeat from core client for 30 sec - exiting 16:11:54 (4496): No heartbeat from core client for 30 sec - exiting 16:11:55 (4496): No heartbeat from core client for 30 sec - exiting 16:11:56 (4496): No heartbeat from core client for 30 sec - exiting 16:11:57 (4496): No heartbeat from core client for 30 sec - exiting 16:11:58 (4496): No heartbeat from core client for 30 sec - exiting 16:11:59 (4496): No heartbeat from core client for 30 sec - exiting 16:12:00 (4496): No heartbeat from core client for 30 sec - exiting 16:12:01 (4496): No heartbeat from core client for 30 sec - exiting 16:12:02 (4496): No heartbeat from core client for 30 sec - exiting 16:12:03 (4496): No heartbeat from core client for 30 sec - exiting 16:12:04 (4496): No heartbeat from core client for 30 sec - exiting 16:12:06 (4496): No heartbeat from core client for 30 sec - exiting 16:12:07 (4496): No heartbeat from core client for 30 sec - exiting 16:12:08 (4496): No heartbeat from core client for 30 sec - exiting 16:12:09 (4496): No heartbeat from core client for 30 sec - exiting 16:12:10 (4496): No heartbeat from core client for 30 sec - exiting 16:12:11 (4496): No heartbeat from core client for 30 sec - exiting 16:12:12 (4496): No heartbeat from core client for 30 sec - exiting 16:12:13 (4496): No heartbeat from core client for 30 sec - exiting 16:12:14 (4496): No heartbeat from core client for 30 sec - exiting 16:12:15 (4496): No heartbeat from core client for 30 sec - exiting 16:12:16 (4496): No heartbeat from core client for 30 sec - exiting 16:12:17 (4496): No heartbeat from core client for 30 sec - exiting 16:12:18 (4496): No heartbeat from core client for 30 sec - exiting 16:12:19 (4496): No heartbeat from core client for 30 sec - exiting 16:12:20 (4496): No heartbeat from core client for 30 sec - exiting 16:12:21 (4496): No heartbeat from core client for 30 sec - exiting 16:12:22 (4496): No heartbeat from core client for 30 sec - exiting 16:12:29 (4496): No heartbeat from core client for 30 sec - exiting 16:12:30 (4496): No heartbeat from core client for 30 sec - exiting 16:12:31 (4496): No heartbeat from core client for 30 sec - exiting 16:12:45 (4496): No heartbeat from core client for 30 sec - exiting 16:12:47 (4496): No heartbeat from core client for 30 sec - exiting 16:12:48 (4496): No heartbeat from core client for 30 sec - exiting 16:12:49 (4496): No heartbeat from core client for 30 sec - exiting 16:12:50 (4496): No heartbeat from core client for 30 sec - exiting 16:12:51 (4496): No heartbeat from core client for 30 sec - exiting 16:12:52 (4496): No heartbeat from core client for 30 sec - exiting 16:12:53 (4496): No heartbeat from core client for 30 sec - exiting 16:12:54 (4496): No heartbeat from core client for 30 sec - exiting 16:12:55 (4496): No heartbeat from core client for 30 sec - exiting 16:12:56 (4496): No heartbeat from core client for 30 sec - exiting 16:12:57 (4496): No heartbeat from core client for 30 sec - exiting 16:12:58 (4496): No heartbeat from core client for 30 sec - exiting 16:12:59 (4496): No heartbeat from core client for 30 sec - exiting 16:13:01 (4496): No heartbeat from core client for 30 sec - exiting 16:13:02 (4496): No heartbeat from core client for 30 sec - exiting 16:13:03 (4496): No heartbeat from core client for 30 sec - exiting 16:13:04 (4496): No heartbeat from core client for 30 sec - exiting 16:13:05 (4496): No heartbeat from core client for 30 sec - exiting 16:13:06 (4496): No heartbeat from core client for 30 sec - exiting 16:13:07 (4496): No heartbeat from core client for 30 sec - exiting 16:13:08 (4496): No heartbeat from core client for 30 sec - exiting 16:13:09 (4496): No heartbeat from core client for 30 sec - exiting 16:13:10 (4496): No heartbeat from core client for 30 sec - exiting 16:13:11 (4496): No heartbeat from core client for 30 sec - exiting 16:13:12 (4496): No heartbeat from core client for 30 sec - exiting 16:13:13 (4496): No heartbeat from core client for 30 sec - exiting 16:13:14 (4496): No heartbeat from core client for 30 sec - exiting 16:13:16 (4496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:31:30 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3676, iMonCtr=2 Model crash detected, will try to restart... 08:29:04 (4864): No heartbeat from core client for 30 sec - exiting 08:29:05 (4864): No heartbeat from core client for 30 sec - exiting 08:29:06 (4864): No heartbeat from core client for 30 sec - exiting 08:29:07 (4864): No heartbeat from core client for 30 sec - exiting 08:29:08 (4864): No heartbeat from core client for 30 sec - exiting 08:29:09 (4864): No heartbeat from core client for 30 sec - exiting 08:29:10 (4864): No heartbeat from core client for 30 sec - exiting 08:29:12 (4864): No heartbeat from core client for 30 sec - exiting 08:29:13 (4864): No heartbeat from core client for 30 sec - exiting 08:29:14 (4864): No heartbeat from core client for 30 sec - exiting 08:29:15 (4864): No heartbeat from core client for 30 sec - exiting 08:29:16 (4864): No heartbeat from core client for 30 sec - exiting 08:29:17 (4864): No heartbeat from core client for 30 sec - exiting 08:29:18 (4864): No heartbeat from core client for 30 sec - exiting 08:29:19 (4864): No heartbeat from core client for 30 sec - exiting 08:29:20 (4864): No heartbeat from core client for 30 sec - exiting 08:29:21 (4864): No heartbeat from core client for 30 sec - exiting 08:29:22 (4864): No heartbeat from core client for 30 sec - exiting 08:29:24 (4864): No heartbeat from core client for 30 sec - exiting 08:29:25 (4864): No heartbeat from core client for 30 sec - exiting 08:29:26 (4864): No heartbeat from core client for 30 sec - exiting 08:29:27 (4864): No heartbeat from core client for 30 sec - exiting 08:29:28 (4864): No heartbeat from core client for 30 sec - exiting 08:29:29 (4864): No heartbeat from core client for 30 sec - exiting 08:29:30 (4864): No heartbeat from core client for 30 sec - exiting 08:29:31 (4864): No heartbeat from core client for 30 sec - exiting 08:29:32 (4864): No heartbeat from core client for 30 sec - exiting 08:29:33 (4864): No heartbeat from core client for 30 sec - exiting 08:29:34 (4864): No heartbeat from core client for 30 sec - exiting 08:29:36 (4864): No heartbeat from core client for 30 sec - exiting 08:29:37 (4864): No heartbeat from core client for 30 sec - exiting 08:29:38 (4864): No heartbeat from core client for 30 sec - exiting 08:29:39 (4864): No heartbeat from core client for 30 sec - exiting 08:29:40 (4864): No heartbeat from core client for 30 sec - exiting 08:29:41 (4864): No heartbeat from core client for 30 sec - exiting 08:29:42 (4864): No heartbeat from core client for 30 sec - exiting 08:29:43 (4864): No heartbeat from core client for 30 sec - exiting 08:29:44 (4864): No heartbeat from core client for 30 sec - exiting 08:29:45 (4864): No heartbeat from core client for 30 sec - exiting 08:29:46 (4864): No heartbeat from core client for 30 sec - exiting 08:29:47 (4864): No heartbeat from core client for 30 sec - exiting 08:29:48 (4864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:29:50 (4864): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=852, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1420, selfPID=4328, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Corker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3376, iMonCtr=2 MonCtr=2 Model crash detected, will try to restart... 06:46:03 (280): No heartbeat from core client for 30 sec - exiting 06:46:04 (280): No heartbeat from core client for 30 sec - exiting 06:46:06 (280): No heartbeat from core client for 30 sec - exiting 06:46:07 (280): No heartbeat from core client for 30 sec - exiting 06:46:08 (280): No heartbeat from core client for 30 sec - exiting 06:46:09 (280): No heartbeat from core client for 30 sec - exiting 06:46:10 (280): No heartbeat from core client for 30 sec - exiting 06:46:11 (280): No heartbeat from core client for 30 sec - exiting 06:46:12 (280): No heartbeat from core client for 30 sec - exiting 06:46:13 (280): No heartbeat from core client for 30 sec - exiting 06:46:14 (280): No heartbeat from core client for 30 sec - exiting 06:46:15 (280): No heartbeat from core client for 30 sec - exiting 06:46:16 (280): No heartbeat from core client for 30 sec - exiting 06:46:18 (280): No heartbeat from core client for 30 sec - exiting 06:46:19 (280): No heartbeat from core client for 30 sec - exiting 06:46:20 (280): No heartbeat from core client for 30 sec - exiting 06:46:21 (280): No heartbeat from core client for 30 sec - exiting 06:46:22 (280): No heartbeat from core client for 30 sec - exiting 06:46:23 (280): No heartbeat from core client for 30 sec - exiting 06:46:24 (280): No heartbeat from core client for 30 sec - exiting 06:46:25 (280): No heartbeat from core client for 30 sec - exiting 06:46:26 (280): No heartbeat from core client for 30 sec - exiting 06:46:27 (280): No heartbeat from core client for 30 sec - exiting 06:46:28 (280): No heartbeat from core client for 30 sec - exiting 06:46:30 (280): No heartbeat from core client for 30 sec - exiting 06:46:31 (280): No heartbeat from core client for 30 sec - exiting 06:46:32 (280): No heartbeat from core client for 30 sec - exiting 06:46:33 (280): No heartbeat from core client for 30 sec - exiting 06:46:34 (280): No heartbeat from core client for 30 sec - exiting 06:46:35 (280): No heartbeat from core client for 30 sec - exiting 06:46:36 (280): No heartbeat from core client for 30 sec - exiting 06:46:37 (280): No heartbeat from core client for 30 sec - exiting 06:46:38 (280): No heartbeat from core client for 30 sec - exiting 06:46:39 (280): No heartbeat from core client for 30 sec - exiting 06:46:40 (280): No heartbeat from core client for 30 sec - exiting 06:46:42 (280): No heartbeat from core client for 30 sec - exiting 06:46:43 (280): No heartbeat from core client for 30 sec - exiting 06:46:44 (280): No heartbeat from core client for 30 sec - exiting 06:46:45 (280): No heartbeat from core client for 30 sec - exiting 06:46:46 (280): No heartbeat from core client for 30 sec - exiting 06:46:47 (280): No heartbeat from core client for 30 sec - exiting 06:46:48 (280): No heartbeat from core client for 30 sec - exiting 06:46:49 (280): No heartbeat from core client for 30 sec - exiting 06:46:50 (280): No heartbeat from core client for 30 sec - exiting 06:46:51 (280): No heartbeat from core client for 30 sec - exiting 06:46:52 (280): No heartbeat from core client for 30 sec - exiting 06:46:54 (280): No heartbeat from core client for 30 sec - exiting 06:46:55 (280): No heartbeat from core client for 30 sec - exiting 06:46:56 (280): No heartbeat from core client for 30 sec - exiting 06:46:57 (280): No heartbeat from core client for 30 sec - exiting 06:46:58 (280): No heartbeat from core client for 30 sec - exiting 06:46:59 (280): No heartbeat from core client for 30 sec - exiting 06:47:00 (280): No heartbeat from core client for 30 sec - exiting 06:47:01 (280): No heartbeat from core client for 30 sec - exiting 06:47:02 (280): No heartbeat from core client for 30 sec - exiting 06:47:03 (280): No heartbeat from core client for 30 sec - exiting 06:47:04 (280): No heartbeat from core client for 30 sec - exiting 06:47:06 (280): No heartbeat from core client for 30 sec - exiting 06:47:07 (280): No heartbeat from core client for 30 sec - exiting 06:47:08 (280): No heartbeat from core client for 30 sec - exiting 06:47:09 (280): No heartbeat from core client for 30 sec - exiting 06:47:10 (280): No heartbeat from core client for 30 sec - exiting 06:47:11 (280): No heartbeat from core client for 30 sec - exiting 06:47:12 (280): No heartbeat from core client for 30 sec - exiting 06:47:13 (280): No heartbeat from core client for 30 sec - exiting 06:47:14 (280): No heartbeat from core client for 30 sec - exiting 06:47:15 (280): No heartbeat from core client for 30 sec - exiting 06:47:17 (280): No heartbeat from core client for 30 sec - exiting 06:47:18 (280): No heartbeat from core client for 30 sec - exiting 06:47:19 (280): No heartbeat from core client for 30 sec - exiting 06:47:20 (280): No heartbeat from core client for 30 sec - exiting 06:47:21 (280): No heartbeat from core client for 30 sec - exiting 06:47:22 (280): No heartbeat from core client for 30 sec - exiting 06:47:23 (280): No heartbeat from core client for 30 sec - exiting 06:47:24 (280): No heartbeat from core client for 30 sec - exiting 06:47:25 (280): No heartbeat from core client for 30 sec - exiting 06:47:26 (280): No heartbeat from core client for 30 sec - exiting 06:47:27 (280): No heartbeat from core client for 30 sec - exiting 06:47:29 (280): No heartbeat from core client for 30 sec - exiting 06:47:30 (280): No heartbeat from core client for 30 sec - exiting 06:47:31 (280): No heartbeat from core client for 30 sec - exiting 06:47:32 (280): No heartbeat from core client for 30 sec - exiting 06:47:33 (280): No heartbeat from core client for 30 sec - exiting 06:47:34 (280): No heartbeat from core client for 30 sec - exiting 06:47:35 (280): No heartbeat from core client for 30 sec - exiting 06:47:36 (280): No heartbeat from core client for 30 sec - exiting 06:47:37 (280): No heartbeat from core client for 30 sec - exiting 06:47:38 (280): No heartbeat from core client for 30 sec - exiting 06:47:39 (280): No heartbeat from core client for 30 sec - exiting 06:47:41 (280): No heartbeat from core client for 30 sec - exiting 06:47:42 (280): No heartbeat from core client for 30 sec - exiting 06:47:43 (280): No heartbeat from core client for 30 sec - exiting 06:47:44 (280): No heartbeat from core client for 30 sec - exiting 06:48:47 (280): No heartbeat from core client for 30 sec - exiting 06:48:48 (280): No heartbeat from core client for 30 sec - exiting 06:48:49 (280): No heartbeat from core client for 30 sec - exiting 06:48:50 (280): No heartbeat from core client for 30 sec - exiting 06:48:51 (280): No heartbeat from core client for 30 sec - exiting 06:48:52 (280): No heartbeat from core client for 30 sec - exiting 06:48:53 (280): No heartbeat from core client for 30 sec - exiting 06:48:54 (280): No heartbeat from core client for 30 sec - exiting 06:48:55 (280): No heartbeat from core client for 30 sec - exiting 06:48:56 (280): No heartbeat from core client for 30 sec - exiting 06:48:58 (280): No heartbeat from core client for 30 sec - exiting 06:48:59 (280): No heartbeat from core client for 30 sec - exiting 06:49:00 (280): No heartbeat from core client for 30 sec - exiting 06:49:01 (280): No heartbeat from core client for 30 sec - exiting 06:49:02 (280): No heartbeat from core client for 30 sec - exiting 06:49:03 (280): No heartbeat from core client for 30 sec - exiting 06:49:04 (280): No heartbeat from core client for 30 sec - exiting 06:49:05 (280): No heartbeat from core client for 30 sec - exiting 06:49:06 (280): No heartbeat from core client for 30 sec - exiting 06:49:07 (280): No heartbeat from core client for 30 sec - exiting 06:49:08 (280): No heartbeat from core client for 30 sec - exiting 06:49:09 (280): No heartbeat from core client for 30 sec - exiting 06:49:10 (280): No heartbeat from core client for 30 sec - exiting 06:49:11 (280): No heartbeat from core client for 30 sec - exiting 06:49:12 (280): No heartbeat from core client for 30 sec - exiting 06:49:13 (280): No heartbeat from core client for 30 sec - exiting 06:49:14 (280): No heartbeat from core client for 30 sec - exiting 06:49:15 (280): No heartbeat from core client for 30 sec - exiting 06:49:16 (280): No heartbeat from core client for 30 sec - exiting 06:49:17 (280): No heartbeat from core client for 30 sec - exiting 06:49:18 (280): No heartbeat from core client for 30 sec - exiting 06:49:19 (280): No heartbeat from core client for 30 sec - exiting 06:49:20 (280): No heartbeat from core client for 30 sec - exiting 06:49:21 (280): No heartbeat from core client for 30 sec - exiting 06:49:22 (280): No heartbeat from core client for 30 sec - exiting 06:49:23 (280): No heartbeat from core client for 30 sec - exiting 06:49:24 (280): No heartbeat from core client for 30 sec - exiting 06:49:25 (280): No heartbeat from core client for 30 sec - exiting 06:49:26 (280): No heartbeat from core client for 30 sec - exiting 06:49:27 (280): No heartbeat from core client for 30 sec - exiting 06:49:29 (280): No heartbeat from core client for 30 sec - exiting 06:49:30 (280): No heartbeat from core client for 30 sec - exiting 06:49:31 (280): No heartbeat from core client for 30 sec - exiting 06:49:32 (280): No heartbeat from core client for 30 sec - exiting 06:49:33 (280): No heartbeat from core client for 30 sec - exiting 06:49:34 (280): No heartbeat from core client for 30 sec - exiting 06:49:35 (280): No heartbeat from core client for 30 sec - exiting 06:49:36 (280): No heartbeat from core client for 30 sec - exiting 06:49:37 (280): No heartbeat from core client for 30 sec - exiting 06:49:38 (280): No heartbeat from core client for 30 sec - exiting 06:49:39 (280): No heartbeat from core client for 30 sec - exiting 06:49:40 (280): No heartbeat from core client for 30 sec - exiting 06:49:41 (280): No heartbeat from core client for 30 sec - exiting 06:49:42 (280): No heartbeat from core client for 30 sec - exiting 06:49:43 (280): No heartbeat from core client for 30 sec - exiting 06:49:44 (280): No heartbeat from core client for 30 sec - exiting 06:49:45 (280): No heartbeat from core client for 30 sec - exiting 06:49:46 (280): No heartbeat from core client for 30 sec - exiting 06:49:48 (280): No heartbeat from core client for 30 sec - exiting 06:49:49 (280): No heartbeat from core client for 30 sec - exiting 06:49:50 (280): No heartbeat from core client for 30 sec - exiting 06:49:51 (280): No heartbeat from core client for 30 sec - exiting 06:49:52 (280): No heartbeat from core client for 30 sec - exiting 06:49:53 (280): No heartbeat from core client for 30 sec - exiting 06:49:54 (280): No heartbeat from core client for 30 sec - exiting 06:49:55 (280): No heartbeat from core client for 30 sec - exiting 06:49:56 (280): No heartbeat from core client for 30 sec - exiting 06:49:57 (280): No heartbeat from core client for 30 sec - exiting 06:49:58 (280): No heartbeat from core client for 30 sec - exiting 06:49:59 (280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3708, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5136, selfPID=2888, iMonCtr=1 Model crash detected, will try to restart... 09:30:47 (4732): No heartbeat from core client for 30 sec - exiting 09:30:48 (4732): No heartbeat from core client for 30 sec - exiting 09:30:49 (4732): No heartbeat from core client for 30 sec - exiting 09:30:50 (4732): No heartbeat from core client for 30 sec - exiting 09:30:51 (4732): No heartbeat from core client for 30 sec - exiting 09:30:52 (4732): No heartbeat from core client for 30 sec - exiting 09:30:53 (4732): No heartbeat from core client for 30 sec - exiting 09:30:54 (4732): No heartbeat from core client for 30 sec - exiting 09:30:55 (4732): No heartbeat from core client for 30 sec - exiting 09:30:56 (4732): No heartbeat from core client for 30 sec - exiting 09:30:57 (4732): No heartbeat from core client for 30 sec - exiting 09:30:59 (4732): No heartbeat from core client for 30 sec - exiting 09:31:00 (4732): No heartbeat from core client for 30 sec - exiting 09:31:01 (4732): No heartbeat from core client for 30 sec - exiting 09:31:02 (4732): No heartbeat from core client for 30 sec - exiting 09:31:03 (4732): No heartbeat from core client for 30 sec - exiting 09:31:04 (4732): No heartbeat from core client for 30 sec - exiting 09:31:05 (4732): No heartbeat from core client for 30 sec - exiting 09:31:06 (4732): No heartbeat from core client for 30 sec - exiting 09:31:07 (4732): No heartbeat from core client for 30 sec - exiting 09:31:08 (4732): No heartbeat from core client for 30 sec - exiting 09:31:09 (4732): No heartbeat from core client for 30 sec - exiting 09:31:11 (4732): No heartbeat from core client for 30 sec - exiting 09:31:12 (4732): No heartbeat from core client for 30 sec - exiting 09:31:13 (4732): No heartbeat from core client for 30 sec - exiting 09:31:14 (4732): No heartbeat from core client for 30 sec - exiting 09:31:15 (4732): No heartbeat from core client for 30 sec - exiting 09:31:16 (4732): No heartbeat from core client for 30 sec - exiting 09:31:17 (4732): No heartbeat from core client for 30 sec - exiting 09:31:18 (4732): No heartbeat from core client for 30 sec - exiting 09:31:19 (4732): No heartbeat from core client for 30 sec - exiting 09:31:20 (4732): No heartbeat from core client for 30 sec - exiting 09:31:21 (4732): No heartbeat from core client for 30 sec - exiting 09:31:22 (4732): No heartbeat from core client for 30 sec - exiting 09:31:23 (4732): No heartbeat from core client for 30 sec - exiting 09:31:24 (4732): No heartbeat from core client for 30 sec - exiting 09:31:25 (4732): No heartbeat from core client for 30 sec - exiting 09:31:27 (4732): No heartbeat from core client for 30 sec - exiting 09:31:28 (4732): No heartbeat from core client for 30 sec - exiting 09:31:29 (4732): No heartbeat from core client for 30 sec - exiting 09:31:30 (4732): No heartbeat from core client for 30 sec - exiting 09:31:31 (4732): No heartbeat from core client for 30 sec - exiting 09:31:32 (4732): No heartbeat from core client for 30 sec - exiting 09:31:33 (4732): No heartbeat from core client for 30 sec - exiting 09:31:34 (4732): No heartbeat from core client for 30 sec - exiting 09:31:35 (4732): No heartbeat from core client for 30 sec - exiting 09:31:36 (4732): No heartbeat from core client for 30 sec - exiting 09:31:37 (4732): No heartbeat from core client for 30 sec - exiting 09:31:38 (4732): No heartbeat from core client for 30 sec - exiting 09:31:39 (4732): No heartbeat from core client for 30 sec - exiting 09:31:40 (4732): No heartbeat from core client for 30 sec - exiting 09:31:42 (4732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5092, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5196, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5400, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Oct 2012 06:34:43	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	138,336	539,287	3.8984
13 Oct 2012 03:16:35	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	126,816	494,990	3.9032
12 Oct 2012 11:55:55	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	115,296	451,662	3.9174
11 Oct 2012 08:35:39	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	103,776	406,645	3.9185
06 Oct 2012 05:26:43	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	92,256	361,281	3.9161
24 Sep 2012 07:44:41	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	80,736	317,012	3.9265
23 Sep 2012 07:50:59	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	69,216	272,340	3.9346
22 Sep 2012 19:16:04	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	57,696	227,521	3.9434
22 Sep 2012 05:57:23	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	46,176	181,726	3.9355
21 Sep 2012 09:17:57	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	34,656	136,139	3.9283
20 Sep 2012 09:31:51	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	23,136	89,854	3.8837
14 Sep 2012 10:07:48	1181641	15221675	hadam3p_pnw_z7s9_1968_1_006940609_1	11,616	44,681	3.8465