Task 15255393

Name	hadam3p_pnw_32dl_1977_1_008192494_0
Workunit	8347618
Created	8 Sep 2012, 17:16:25 UTC
Sent	9 Sep 2012, 0:27:18 UTC
Report deadline	22 Aug 2013, 5:47:18 UTC
Received	16 Oct 2012, 21:16:24 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1181312
Run time	5 days 16 hours 8 min 32 sec
CPU time	5 days 5 hours 28 min 13 sec
Validate state	Workunit error - check skipped
Credit	3,005.88
Device peak FLOPS	2.24 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4504, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 18:06:22 (3776): No heartbeat from core client for 30 sec - exiting 18:06:24 (3776): No heartbeat from core client for 30 sec - exiting 18:06:25 (3776): No heartbeat from core client for 30 sec - exiting 18:06:26 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6388, selfPID=4948, iMonCtr=1 Model crash detected, will try to restart... 14:18:19 (3900): No heartbeat from core client for 30 sec - exiting 14:18:20 (3900): No heartbeat from core client for 30 sec - exiting 14:18:21 (3900): No heartbeat from core client for 30 sec - exiting 14:18:22 (3900): No heartbeat from core client for 30 sec - exiting 14:18:23 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not runn16:15:57 (3996): No heartbeat from core client for 30 sec - exiting 16:15:58 (3996): No heartbeat from core client for 30 sec - exiting 16:15:59 (3996): No heartbeat from core client for 30 sec - exiting 16:16:00 (3996): No heartbeat from core client for 30 sec - exiting 16:16:01 (3996): No heartbeat from core client for 30 sec - exiting 16:16:02 (3996): No heartbeat from core client for 30 sec - exiting 16:16:03 (3996): No heartbeat from core client for 30 sec - exiting 16:16:05 (3996): No heartbeat from core client for 30 sec - exiting 16:16:06 (3996): No heartbeat from core client for 30 sec - exiting 16:16:07 (3996): No heartbeat from core client for 30 sec - exiting 16:16:08 (3996): No heartbeat from core client for 30 sec - exiting 16:16:09 (3996): No heartbeat from core client for 30 sec - exiting 16:16:10 (3996): No heartbeat from core client for 30 sec - exiting 16:16:11 (3996): No heartbeat from core client for 30 sec - exiting 16:16:12 (3996): No heartbeat from core client for 30 sec - exiting 16:16:13 (3996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3984, selfPID=5888, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4768, selfPID=7172, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4708, selfPID=3552, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5268, selfPID=5268, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:29:35 (3724): No heartbeat from core client for 30 sec - exiting 23:29:36 (3724): No heartbeat from core client for 30 sec - exiting 23:29:37 (3724): No heartbeat from core client for 30 sec - exiting 23:29:38 (3724): No heartbeat from core client for 30 sec - exiting 23:29:39 (3724): No heartbeat from core client for 30 sec - exiting 23:29:40 (3724): No heartbeat from core client for 30 sec - exiting 23:29:41 (3724): No heartbeat from core client for 30 sec - exiting 23:29:43 (3724): No heartbeat from core client for 30 sec - exiting 23:29:44 (3724): No heartbeat from core client for 30 sec - exiting 23:29:45 (3724): No heartbeat from core client for 30 sec - exiting 23:29:46 (3724): No heartbeat from core client for 30 sec - exiting 23:29:47 (3724): No heartbeat from core client for 30 sec - exiting 23:29:48 (3724): No heartbeat from core client for 30 sec - exiting 23:29:49 (3724): No heartbeat from core client for 30 sec - exiting 23:29:50 (3724): No heartbeat from core client for 30 sec - exiting 23:29:51 (3724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3956, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 08:25:38 (3776): No heartbeat from core client for 30 sec - exiting 08:25:39 (3776): No heartbeat from core client for 30 sec - exiting 08:25:40 (3776): No heartbeat from core client for 30 sec - exiting 08:25:41 (3776): No heartbeat from core client for 30 sec - exiting 08:25:42 (3776): No heartbeat from core client for 30 sec - exiting 08:25:43 (3776): No heartbeat from core client for 30 sec - exiting 08:25:45 (3776): No heartbeat from core client for 30 sec - exiting 08:25:46 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:51:22 (3784): No heartbeat from core client for 30 sec - exiting 11:51:23 (3784): No heartbeat from core client for 30 sec - exiting 11:51:24 (3784): No heartbeat from core client for 30 sec - exiting 11:51:25 (3784): No heartbeat from core client for 30 sec - exiting 11:51:26 (3784): No heartbeat from core client for 30 sec - exiting 11:51:27 (3784): No heartbeat from core client for 30 sec - exiting 11:51:28 (3784): No heartbeat from core client for 30 sec - exiting 11:51:29 (3784): No heartbeat from core client for 30 sec - exiting 11:51:30 (3784): No heartbeat from core client for 30 sec - exiting 11:51:31 (3784): No heartbeat from core client for 30 sec - exiting 11:51:33 (3784): No heartbeat from core client for 30 sec - exiting 11:51:34 (3784): No heartbeat from core client for 30 sec - exiting 11:51:35 (3784): No heartbeat from core client for 30 sec - exiting 11:51:36 (3784): No heartbeat from core client for 30 sec - exiting 11:51:37 (3784): No heartbeat from core client for 30 sec - exiting 11:51:38 (3784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4428, iMonCtr=2 Model crash detected, will try to restart... 17:25:29 (3816): No heartbeat from core client for 30 sec - exiting 17:25:30 (3816): No heartbeat from core client for 30 sec - exiting 17:25:31 (3816): No heartbeat from core client for 30 sec - exiting 17:25:32 (3816): No heartbeat from core client for 30 sec - exiting 17:25:33 (3816): No heartbeat from core client for 30 sec - exiting 17:25:34 (3816): No heartbeat from core client for 30 sec - exiting 17:25:35 (3816): No heartbeat from core client for 30 sec - exiting 17:25:36 (3816): No heartbeat from core client for 30 sec - exiting 17:25:38 (3816): No heartbeat from core client for 30 sec - exiting 17:25:39 (3816): No heartbeat from core client for 30 sec - exiting 17:25:40 (3816): No heartbeat from core client for 30 sec - exiting 17:25:41 (3816): No heartbeat from core client for 30 sec - exiting 17:25:42 (3816): No heartbeat from core client for 30 sec - exiting 17:25:43 (3816): No heartbeat from core client for 30 sec - exiting 17:25:44 (3816): No heartbeat from core client for 30 sec - exiting 17:25:45 (3816): No heartbeat from core client for 30 sec - exiting 17:25:46 (3816): No heartbeat from core client for 30 sec - exiting 17:25:47 (3816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:47:10 (5328): No heartbeat from core client for 30 sec - exiting 20:47:11 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:09:54 (4032): No heartbeat from core client for 30 sec - exiting 09:09:55 (4032): No heartbeat from core client for 30 sec - exiting 09:09:56 (4032): No heartbeat from core client for 30 sec - exiting 09:09:57 (4032): No heartbeat from core client for 30 sec - exiting 09:09:58 (4032): No heartbeat from core client for 30 sec - exiting 09:09:59 (4032): No heartbeat from core client for 30 sec - exiting 09:10:00 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:33:03 (3340): No heartbeat from core client for 30 sec - exiting 19:33:04 (3340): No heartbeat from core client for 30 sec - exiting 19:33:05 (3340): No heartbeat from core client for 30 sec - exiting 19:33:07 (3340): No heartbeat from core client for 30 sec - exiting 19:33:08 (3340): No heartbeat from core client for 30 sec - exiting 19:33:09 (3340): No heartbeat from core client for 30 sec - exiting 19:33:10 (3340): No heartbeat from core client for 30 sec - exiting 19:33:11 (3340): No heartbeat from core client for 30 sec - exiting 19:33:12 (3340): No heartbeat from core client for 30 sec - exiting 19:33:13 (3340): No heartbeat from core client for 30 sec - exiting 19:33:14 (3340): No heartbeat from core client for 30 sec - exiting 19:33:15 (3340): No heartbeat from core client for 30 sec - exiting 19:33:16 (3340): No heartbeat from core client for 30 sec - exiting 19:33:17 (3340): No heartbeat from core client for 30 sec - exiting 19:33:18 (3340): No heartbeat from core client for 30 sec - exiting 19:33:20 (3340): No heartbeat from core client for 30 sec - exiting 19:33:21 (3340): No heartbeat from core client for 30 sec - exiting 19:33:22 (3340): No heartbeat from core client for 30 sec - exiting 19:33:23 (3340): No heartbeat from core client for 30 sec - exiting 19:33:24 (3340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:15:04 (3496): No heartbeat from core client for 30 sec - exiting 22:15:05 (3496): No heartbeat from core client for 30 sec - exiting 22:15:06 (3496): No heartbeat from core client for 30 sec - exiting 22:15:07 (3496): No heartbeat from core client for 30 sec - exiting 22:15:08 (3496): No heartbeat from core client for 30 sec - exiting 22:15:09 (3496): No heartbeat from core client for 30 sec - exiting 22:15:10 (3496): No heartbeat from core client for 30 sec - exiting 22:15:11 (3496): No heartbeat from core client for 30 sec - exiting 22:15:12 (3496): No heartbeat from core client for 30 sec - exiting 22:15:14 (3496): No heartbeat from core client for 30 sec - exiting 22:15:15 (3496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:41:00 (3924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:48:08 (3196): No heartbeat from core client for 30 sec - exiting 09:48:09 (3196): No heartbeat from core client for 30 sec - exiting 09:48:10 (3196): No heartbeat from core client for 30 sec - exiting 09:48:11 (3196): No heartbeat from core client for 30 sec - exiting 09:48:13 (3196): No heartbeat from core client for 30 sec - exiting 09:48:14 (3196): No heartbeat from core client for 30 sec - exiting 09:48:15 (3196): No heartbeat from core client for 30 sec - exiting 09:48:16 (3196): No heartbeat from core client for 30 sec - exiting 09:48:17 (3196): No heartbeat from core client for 30 sec - exiting 09:48:18 (3196): No heartbeat from core client for 30 sec - exiting 09:48:19 (3196): No heartbeat from core client for 30 sec - exiting 09:48:20 (3196): No heartbeat from core client for 30 sec - exiting 09:48:21 (3196): No heartbeat from core client for 30 sec - exiting 09:48:22 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4980, selfPID=4980, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:15:02 (4196): No heartbeat from core client for 30 sec - exiting 23:15:03 (4196): No heartbeat from core client for 30 sec - exiting 23:15:04 (4196): No heartbeat from core client for 30 sec - exiting 23:15:06 (4196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:42:28 (3592): No heartbeat from core client for 30 sec - exiting 07:42:29 (3592): No heartbeat from core client for 30 sec - exiting 07:42:30 (3592): No heartbeat from core client for 30 sec - exiting 07:42:31 (3592): No heartbeat from core client for 30 sec - exiting 07:42:32 (3592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:33:37 (3524): No heartbeat from core client for 30 sec - exiting 18:33:38 (3524): No heartbeat from core client for 30 sec - exiting 18:33:40 (3524): No heartbeat from core client for 30 sec - exiting 18:33:41 (3524): No heartbeat from core client for 30 sec - exiting 18:33:42 (3524): No heartbeat from core client for 30 sec - exiting 18:33:43 (3524): No heartbeat from core client for 30 sec - exiting 18:33:44 (3524): No heartbeat from core client for 30 sec - exiting 18:33:45 (3524): No heartbeat from core client for 30 sec - exiting 18:33:46 (3524): No heartbeat from core client for 30 sec - exiting 18:33:47 (3524): No heartbeat from core client for 30 sec - exiting 18:33:48 (3524): No heartbeat from core client for 30 sec - exiting 18:33:49 (3524): No heartbeat from core client for 30 sec - exiting 18:33:50 (3524): No heartbeat from core client for 30 sec - exiting 18:33:52 (3524): No heartbeat from core client for 30 sec - exiting 18:33:53 (3524): No heartbeat from core client for 30 sec - exiting 18:33:54 (3524): No heartbeat from core client for 30 sec - exiting 18:33:55 (3524): No heartbeat from core client for 30 sec - exiting 18:33:56 (3524): No heartbeat from core client for 30 sec - exiting 18:33:57 (3524): No heartbeat from core client for 30 sec - exiting 18:33:58 (3524): No heartbeat from core client for 30 sec - exiting 18:33:59 (3524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4872, selfPID=4084, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7712, selfPID=7712, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 01:10:41 (3720): No heartbeat from core client for 30 sec - exiting 01:10:42 (3720): No heartbeat from core client for 30 sec - exiting 01:10:43 (3720): No heartbeat from core client for 30 sec - exiting 01:10:44 (3720): No heartbeat from core client for 30 sec - exiting 01:10:45 (3720): No heartbeat from core client for 30 sec - exiting 01:10:46 (3720): No heartbeat from core client for 30 sec - exiting 01:10:47 (3720): No heartbeat from core client for 30 sec - exiting 01:10:48 (3720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4788, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2380, iMonCtr=2 15:52:19 (3696): No heartbeat from core client for 30 sec - exiting 15:52:20 (3696): No heartbeat from core client for 30 sec - exiting 15:52:21 (3696): No heartbeat from core client for 30 sec - exiting 15:52:22 (3696): No heartbeat from core client for 30 sec - exiting 15:52:23 (3696): No heartbeat from core client for 30 sec - exiting 15:52:25 (3696): No heartbeat from core client for 30 sec - exiting 15:52:26 (3696): No heartbeat from core client for 30 sec - exiting 15:52:27 (3696): No heartbeat from core client for 30 sec - exiting 15:52:28 (3696): No heartbeat from core client for 30 sec - exiting 15:52:29 (3696): No heartbeat from core client for 30 sec - exiting 15:52:30 (3696): No heartbeat from core client for 30 sec - exiting 15:52:31 (3696): No heartbeat from core client for 30 sec - exiting 15:52:32 (3696): No heartbeat from core client for 30 sec - exiting 15:52:33 (3696): No heartbeat from core client for 30 sec - exiting 15:52:34 (3696): No heartbeat from core client for 30 sec - exiting 15:52:35 (3696): No heartbeat from core client for 30 sec - exiting 15:52:37 (3696): No heartbeat from core client for 30 sec - exiting 15:52:38 (3696): No heartbeat from core client for 30 sec - exiting 15:52:39 (3696): No heartbeat from core client for 30 sec - exiting 15:52:40 (3696): No heartbeat from core client for 30 sec - exiting 15:52:41 (3696): No heartbeat from core client for 30 sec - exiting 15:52:42 (3696): No heartbeat from core client for 30 sec - exiting 15:52:43 (3696): No heartbeat from core client for 30 sec - exiting 15:52:44 (3696): No heartbeat from core client for 30 sec - exiting 15:52:45 (3696): No heartbeat from core client for 30 sec - exiting 15:52:46 (3696): No heartbeat from core client for 30 sec - exiting 15:52:47 (3696): No heartbeat from core client for 30 sec - exiting 15:52:49 (3696): No heartbeat from core client for 30 sec - exiting 15:52:50 (3696): No heartbeat from core client for 30 sec - exiting 15:52:51 (3696): No heartbeat from core client for 30 sec - exiting 15:52:52 (3696): No heartbeat from core client for 30 sec - exiting 15:52:53 (3696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:43:55 (2840): No heartbeat from core client for 30 sec - exiting 08:43:56 (2840): No heartbeat from core client for 30 sec - exiting 08:43:57 (2840): No heartbeat from core client for 30 sec - exiting 08:43:58 (2840): No heartbeat from core client for 30 sec - exiting 08:43:59 (2840): No heartbeat from core client for 30 sec - exiting 08:44:00 (2840): No heartbeat from core client for 30 sec - exiting 08:44:02 (2840): No heartbeat from core client for 30 sec - exiting 08:44:03 (2840): No heartbeat from core client for 30 sec - exiting 08:44:04 (2840): No heartbeat from core client for 30 sec - exiting 08:44:05 (2840): No heartbeat from core client for 30 sec - exiting 08:44:06 (2840): No heartbeat from core client for 30 sec - exiting 08:44:07 (2840): No heartbeat from core client for 30 sec - exiting 08:44:08 (2840): No heartbeat from core client for 30 sec - exiting 08:44:09 (2840): No heartbeat from core client for 30 sec - exiting 08:44:10 (2840): No heartbeat from core client for 30 sec - exiting 08:44:11 (2840): No heartbeat from core client for 30 sec - exiting 08:44:12 (2840): No heartbeat from core client for 30 sec - exiting 08:44:14 (2840): No heartbeat from core client for 30 sec - exiting 08:44:15 (2840): No heartbeat from core client for 30 sec - exiting 08:44:16 (2840): No heartbeat from core client for 30 sec - exiting 08:44:17 (2840): No heartbeat from core client for 30 sec - exiting 08:44:18 (2840): No heartbeat from core client for 30 sec - exiting 08:44:19 (2840): No heartbeat from core client for 30 sec - exiting 08:44:20 (2840): No heartbeat from core client for 30 sec - exiting 08:44:21 (2840): No heartbeat from core client for 30 sec - exiting 08:44:22 (2840): No heartbeat from core client for 30 sec - exiting 08:44:23 (2840): No heartbeat from core client for 30 sec - exiting 08:44:24 (2840): No heartbeat from core client for 30 sec - exiting 08:44:26 (2840): No heartbeat from core client for 30 sec - exiting 08:44:27 (2840): No heartbeat from core client for 30 sec - exiting 08:44:28 (2840): No heartbeat from core client for 30 sec - exiting 08:44:29 (2840): No heartbeat from core client for 30 sec - exiting 08:44:30 (2840): No heartbeat from core client for 30 sec - exiting 08:44:31 (2840): No heartbeat from core client for 30 sec - exiting 08:44:32 (2840): No heartbeat from core client for 30 sec - exiting 08:44:33 (2840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:49:52 (4980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6540, selfPID=6540, iMonCtr=2 CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6332, selfPID=5596, iMonCtr=1 Model crash detected, will try to restart... 08:54:51 (3112): No heartbeat from core client for 30 sec - exiting 08:54:52 (3112): No heartbeat from core client for 30 sec - exiting 08:54:53 (3112): No heartbeat from core client for 30 sec - exiting 08:54:54 (3112): No heartbeat from core client for 30 sec - exiting 08:54:55 (3112): No heartbeat from core client for 30 sec - exiting 08:54:56 (3112): No heartbeat from core client for 30 sec - exiting 08:54:57 (3112): No heartbeat from core client for 30 sec - exiting 08:54:58 (3112): No heartbeat from core client for 30 sec - exiting 08:54:59 (3112): No heartbeat from core client for 30 sec - exiting 08:55:00 (3112): No heartbeat from core client for 30 sec - exiting 08:55:02 (3112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:05:31 (3212): No heartbeat from core client for 30 sec - exiting 19:05:32 (3212): No heartbeat from core client for 30 sec - exiting 19:05:33 (3212): No heartbeat from core client for 30 sec - exiting 19:05:34 (3212): No heartbeat from core client for 30 sec - exiting 19:05:35 (3212): No heartbeat from core client for 30 sec - exiting 19:05:36 (3212): No heartbeat from core client for 30 sec - exiting 19:05:37 (3212): No heartbeat from core client for 30 sec - exiting 19:05:38 (3212): No heartbeat from core client for 30 sec - exiting 19:05:39 (3212): No heartbeat from core client for 30 sec - exiting 19:05:40 (3212): No heartbeat from core client for 30 sec - exiting 19:05:41 (3212): No heartbeat from core client for 30 sec - exiting 19:05:42 (3212): No heartbeat from core client for 30 sec - exiting 19:05:44 (3212): No heartbeat from core client for 30 sec - exiting 19:05:45 (3212): No heartbeat from core client for 30 sec - exiting 19:05:46 (3212): No heartbeat from core client for 30 sec - exiting 19:05:47 (3212): No heartbeat from core client for 30 sec - exiting 19:05:48 (3212): No heartbeat from core client for 30 sec - exiting 19:05:49 (3212): No heartbeat from core client for 30 sec - exiting 19:05:50 (3212): No heartbeat from core client for 30 sec - exiting 19:05:51 (3212): No heartbeat from core client for 30 sec - exiting 19:05:52 (3212): No heartbeat from core client for 30 sec - exiting 19:05:53 (3212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:03:23 (3468): No heartbeat from core client for 30 sec - exiting 08:03:24 (3468): No heartbeat from core client for 30 sec - exiting 08:03:25 (3468): No heartbeat from core client for 30 sec - exiting 08:03:26 (3468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:20:13 (3756): No heartbeat from core client for 30 sec - exiting 18:20:14 (3756): No heartbeat from core client for 30 sec - exiting 18:20:15 (3756): No heartbeat from core client for 30 sec - exiting 18:20:16 (3756): No heartbeat from core client for 30 sec - exiting 18:20:17 (3756): No heartbeat from core client for 30 sec - exiting 18:20:18 (3756): No heartbeat from core client for 30 sec - exiting 18:20:19 (3756): No heartbeat from core client for 30 sec - exiting 18:20:21 (3756): No heartbeat from core client for 30 sec - exiting 18:20:22 (3756): No heartbeat from core client for 30 sec - exiting 18:20:23 (3756): No heartbeat from core client for 30 sec - exiting 18:20:24 (3756): No heartbeat from core client for 30 sec - exiting 18:20:25 (3756): No heartbeat from core client for 30 sec - exiting 18:20:26 (3756): No heartbeat from core client for 30 sec - exiting 18:20:27 (3756): No heartbeat from core client for 30 sec - exiting 18:20:28 (3756): No heartbeat from core client for 30 sec - exiting 18:20:29 (3756): No heartbeat from core client for 30 sec - exiting 18:20:30 (3756): No heartbeat from core client for 30 sec - exiting 18:20:31 (3756): No heartbeat from core client for 30 sec - exiting 18:20:33 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7636, selfPID=8048, iMonCtr=1 Model crash detected, will try to restart... 12:22:59 (2064): No heartbeat from core client for 30 sec - exiting 12:23:00 (2064): No heartbeat from core client for 30 sec - exiting 12:23:01 (2064): No heartbeat from core client for 30 sec - exiting 12:23:03 (2064): No heartbeat from core client for 30 sec - exiting 12:23:04 (2064): No heartbeat from core client for 30 sec - exiting 12:23:05 (2064): No heartbeat from core client for 30 sec - exiting 12:23:06 (2064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6556, selfPID=6556, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 20:48:06 (3652): No heartbeat from core client for 30 sec - exiting 20:48:07 (3652): No heartbeat from core client for 30 sec - exiting 20:48:08 (3652): No heartbeat from core client for 30 sec - exiting 20:48:09 (3652): No heartbeat from core client for 30 sec - exiting 20:48:10 (3652): No heartbeat from core client for 30 sec - exiting 20:48:11 (3652): No heartbeat from core client for 30 sec - exiting 20:48:12 (3652): No heartbeat from core client for 30 sec - exiting 20:48:13 (3652): No heartbeat from core client for 30 sec - exiting 20:48:14 (3652): No heartbeat from core client for 30 sec - exiting 20:48:15 (3652): No heartbeat from core client for 30 sec - exiting 20:48:16 (3652): No heartbeat from core client for 30 sec - exiting 20:48:17 (3652): No heartbeat from core client for 30 sec - exiting 20:48:18 (3652): No heartbeat from core client for 30 sec - exiting 20:48:19 (3652): No heartbeat from core client for 30 sec - exiting 20:48:20 (3652): No heartbeat from core client for 30 sec - exiting 20:48:21 (3652): No heartbeat from core client for 30 sec - exiting 20:48:22 (3652): No heartbeat from core client for 30 sec - exiting 20:48:23 (3652): No heartbeat from core client for 30 sec - exiting 20:48:24 (3652): No heartbeat from core client for 30 sec - exiting 20:48:25 (3652): No heartbeat from core client for 30 sec - exiting 20:48:26 (3652): No heartbeat from core client for 30 sec - exiting 20:48:27 (3652): No heartbeat from core client for 30 sec - exiting 20:48:28 (3652): No heartbeat from core client for 30 sec - exiting 20:48:29 (3652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:50:58 (2596): No heartbeat from core client for 30 sec - exiting 22:50:59 (2596): No heartbeat from core client for 30 sec - exiting 22:51:00 (2596): No heartbeat from core client for 30 sec - exiting 22:51:01 (2596): No heartbeat from core client for 30 sec - exiting 22:51:02 (2596): No heartbeat from core client for 30 sec - exiting 22:51:03 (2596): No heartbeat from core client for 30 sec - exiting 22:51:04 (2596): No heartbeat from core client for 30 sec - exiting 22:51:05 (2596): No heartbeat from core client for 30 sec - exiting 22:51:06 (2596): No heartbeat from core client for 30 sec - exiting 22:51:07 (2596): No heartbeat from core client for 30 sec - exiting 22:51:08 (2596): No heartbeat from core client for 30 sec - exiting 22:51:09 (2596): No heartbeat from core client for 30 sec - exiting 22:51:10 (2596): No heartbeat from core client for 30 sec - exiting 22:51:11 (2596): No heartbeat from core client for 30 sec - exiting 22:51:12 (2596): No heartbeat from core client for 30 sec - exiting 22:51:13 (2596): No heartbeat from core client for 30 sec - exiting 22:51:14 (2596): No heartbeat from core client for 30 sec - exiting 22:51:15 (2596): No heartbeat from core client for 30 sec - exiting 22:51:16 (2596): No heartbeat from core client for 30 sec - exiting 22:51:17 (2596): No heartbeat from core client for 30 sec - exiting 22:51:18 (2596): No heartbeat from core client for 30 sec - exiting 22:51:19 (2596): No heartbeat from core client for 30 sec - exiting 22:51:20 (2596): No heartbeat from core client for 30 sec - exiting 22:51:21 (2596): No heartbeat from core client for 30 sec - exiting 22:51:22 (2596): No heartbeat from core client for 30 sec - exiting 22:51:23 (2596): No heartbeat from core client for 30 sec - exiting 22:51:24 (2596): No heartbeat from core client for 30 sec - exiting 22:51:25 (2596): No heartbeat from core client for 30 sec - exiting 22:51:26 (2596): No heartbeat from core client for 30 sec - exiting 22:51:27 (2596): No heartbeat from core client for 30 sec - exiting 22:51:28 (2596): No heartbeat from core client for 30 sec - exiting 22:52:02 (2596): No heartbeat from core client for 30 sec - exiting 22:52:03 (2596): No heartbeat from core client for 30 sec - exiting 22:52:04 (2596): No heartbeat from core client for 30 sec - exiting 22:52:05 (2596): No heartbeat from core client for 30 sec - exiting 22:52:06 (2596): No heartbeat from core client for 30 sec - exiting 22:52:07 (2596): No heartbeat from core client for 30 sec - exiting 22:52:08 (2596): No heartbeat from core client for 30 sec - exiting 22:52:09 (2596): No heartbeat from core client for 30 sec - exiting 22:52:10 (2596): No heartbeat from core client for 30 sec - exiting 22:52:11 (2596): No heartbeat from core client for 30 sec - exiting 22:52:12 (2596): No heartbeat from core client for 30 sec - exiting 22:52:13 (2596): No heartbeat from core client for 30 sec - exiting 22:52:14 (2596): No heartbeat from core client for 30 sec - exiting 22:52:15 (2596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:52:16 (2596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6360, selfPID=6360, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 16:09:04 (6296): No heartbeat from core client for 30 sec - exiting 16:09:05 (6296): No heartbeat from core client for 30 sec - exiting 16:09:06 (6296): No heartbeat from core client for 30 sec - exiting 16:09:07 (6296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:32:41 (5680): No heartbeat from core client for 30 sec - exiting 20:32:42 (5680): No heartbeat from core client for 30 sec - exiting 20:32:43 (5680): No heartbeat from core client for 30 sec - exiting 20:32:44 (5680): No heartbeat from core client for 30 sec - exiting 20:32:46 (5680): No heartbeat from core client for 30 sec - exiting 20:32:47 (5680): No heartbeat from core client for 30 sec - exiting 20:32:48 (5680): No heartbeat from core client for 30 sec - exiting 20:32:49 (5680): No heartbeat from core client for 30 sec - exiting 20:32:50 (5680): No heartbeat from core client for 30 sec - exiting 20:32:51 (5680): No heartbeat from core client for 30 sec - exiting 20:32:52 (5680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:45:50 (4048): No heartbeat from core client for 30 sec - exiting 13:45:52 (4048): No heartbeat from core client for 30 sec - exiting 13:45:53 (4048): No heartbeat from core client for 30 sec - exiting 13:45:54 (4048): No heartbeat from core client for 30 sec - exiting 13:45:55 (4048): No heartbeat from core client for 30 sec - exiting 13:45:56 (4048): No heartbeat from core client for 30 sec - exiting 13:45:57 (4048): No heartbeat from core client for 30 sec - exiting 13:45:58 (4048): No heartbeat from core client for 30 sec - exiting 13:45:59 (4048): No heartbeat from core client for 30 sec - exiting 13:46:00 (4048): No heartbeat from core client for 30 sec - exiting 13:46:01 (4048): No heartbeat from core client for 30 sec - exiting 13:46:02 (4048): No heartbeat from core client for 30 sec - exiting 13:46:04 (4048): No heartbeat from core client for 30 sec - exiting 13:46:05 (4048): No heartbeat from core client for 30 sec - exiting 13:46:06 (4048): No heartbeat from core client for 30 sec - exiting 13:46:07 (4048): No heartbeat from core client for 30 sec - exiting 13:46:08 (4048): No heartbeat from core client for 30 sec - exiting 13:46:09 (4048): No heartbeat from core client for 30 sec - exiting 13:46:10 (4048): No heartbeat from core client for 30 sec - exiting 13:46:11 (4048): No heartbeat from core client for 30 sec - exiting 13:46:12 (4048): No heartbeat from core client for 30 sec - exiting 13:46:13 (4048): No heartbeat from core client for 30 sec - exiting 13:46:15 (4048): No heartbeat from core client for 30 sec - exiting 13:46:16 (4048): No heartbeat from core client for 30 sec - exiting 13:46:17 (4048): No heartbeat from core client for 30 sec - exiting 13:46:18 (4048): No heartbeat from core client for 30 sec - exiting 13:46:19 (4048): No heartbeat from core client for 30 sec - exiting 13:46:20 (4048): No heartbeat from core client for 30 sec - exiting 13:46:21 (4048): No heartbeat from core client for 30 sec - exiting 13:46:22 (4048): No heartbeat from core client for 30 sec - exiting 13:46:23 (4048): No heartbeat from core client for 30 sec - exiting 13:46:24 (4048): No heartbeat from core client for 30 sec - exiting 13:46:25 (4048): No heartbeat from core client for 30 sec - exiting 13:46:27 (4048): No heartbeat from core client for 30 sec - exiting 13:46:28 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2696, selfPID=3764, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Oct 2012 14:45:44	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	138,336	450,947	3.2598
13 Oct 2012 02:06:24	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	126,817	412,158	3.2500
13 Oct 2012 00:56:12	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	126,816	411,674	3.2462
10 Oct 2012 18:34:41	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	115,296	372,518	3.2310
08 Oct 2012 13:47:05	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	103,776	334,102	3.2195
06 Oct 2012 22:52:08	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	92,256	297,746	3.2274
04 Oct 2012 21:44:23	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	80,738	260,966	3.2323
04 Oct 2012 20:33:26	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	80,736	260,476	3.2263
02 Oct 2012 18:30:15	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	69,216	224,893	3.2491
26 Sep 2012 01:00:55	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	57,696	187,110	3.2430
23 Sep 2012 02:24:45	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	46,176	149,890	3.2461
20 Sep 2012 23:14:55	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	34,656	112,974	3.2599
16 Sep 2012 01:57:53	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	23,136	75,772	3.2751
14 Sep 2012 01:10:32	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	11,618	38,179	3.2862
13 Sep 2012 12:18:06	1181312	15255393	hadam3p_pnw_32dl_1977_1_008192494_0	11,616	37,724	3.2476