Task 15832067

Name	hadcm3n_o09k_1980_40_008388707_1
Workunit	8539566
Created	6 Jun 2013, 2:22:24 UTC
Sent	6 Jun 2013, 2:51:22 UTC
Report deadline	5 Sep 2013, 10:18:33 UTC
Received	5 Jul 2013, 4:01:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1189257
Run time	9 days 9 hours 18 min 56 sec
CPU time	2 days 11 hours 11 min
Validate state	Invalid
Credit	933.12
Device peak FLOPS	2.09 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4680, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:47:36 (6576): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:21:17 (3152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:40 (8896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:41 (8896): No heartbeat from core client for 30 sec - exiting 20:45:42 (8896): No heartbeat from core client for 30 sec - exiting 20:45:43 (8896): No heartbeat from core client for 30 sec - exiting 20:57:54 (7976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:57:55 (7976): No heartbeat from core client for 30 sec - exiting 20:57:56 (7976): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6820, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:49:38 (7388): No heartbeat from core client for 30 sec - exiting 19:49:39 (7388): No heartbeat from core client for 30 sec - exiting 19:49:40 (7388): No heartbeat from core client for 30 sec - exiting 19:49:41 (7388): No heartbeat from core client for 30 sec - exiting 19:49:42 (7388): No heartbeat from core client for 30 sec - exiting 19:49:43 (7388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:07:54 (7772): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8864, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 13:22:36 (4876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:22:37 (4876): No heartbeat from core client for 30 sec - exiting 13:22:38 (4876): No heartbeat from core client for 30 sec - exiting 13:22:39 (4876): No heartbeat from core client for 30 sec - exiting 13:22:40 (4876): No heartbeat from core client for 30 sec - exiting 13:22:41 (4876): No heartbeat from core client for 30 sec - exiting 13:55:54 (9000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:55:55 (9000): No heartbeat from core client for 30 sec - exiting 13:55:56 (9000): No heartbeat from core client for 30 sec - exiting 13:55:57 (9000): No heartbeat from core client for 30 sec - exiting 13:55:58 (9000): No heartbeat from core client for 30 sec - exiting 13:55:59 (9000): No heartbeat from core client for 30 sec - exiting 17:55:30 (9300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:55:31 (9300): No heartbeat from core client for 30 sec - exiting 17:55:32 (9300): No heartbeat from core client for 30 sec - exiting 17:55:33 (9300): No heartbeat from core client for 30 sec - exiting 17:55:34 (9300): No heartbeat from core client for 30 sec - exiting 17:55:35 (9300): No heartbeat from core client for 30 sec - exiting 17:55:36 (9300): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 08:46:04 (3388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:46:05 (3388): No heartbeat from core client for 30 sec - exiting 08:50:10 (9352): No heartbeat from core client for 30 sec - exiting 08:50:11 (9352): No heartbeat from core client for 30 sec - exiting 08:50:12 (9352): No heartbeat from core client for 30 sec - exiting 08:50:13 (9352): No heartbeat from core client for 30 sec - exiting 08:50:14 (9352): No heartbeat from core client for 30 sec - exiting 08:50:15 (9352): No heartbeat from core client for 30 sec - exiting 08:50:16 (9352): No heartbeat from core client for 30 sec - exiting 08:50:17 (9352): No heartbeat from core client for 30 sec - exiting 08:50:18 (9352): No heartbeat from core client for 30 sec - exiting 08:50:19 (9352): No heartbeat from core client for 30 sec - exiting 08:50:20 (9352): No heartbeat from core client for 30 sec - exiting 08:50:21 (9352): No heartbeat from core client for 30 sec - exiting 08:50:22 (9352): No heartbeat from core client for 30 sec - exiting 08:50:23 (9352): No heartbeat from core client for 30 sec - exiting 08:50:24 (9352): No heartbeat from core client for 30 sec - exiting 08:50:25 (9352): No heartbeat from core client for 30 sec - exiting 08:50:26 (9352): No heartbeat from core client for 30 sec - exiting 08:50:27 (9352): No heartbeat from core client for 30 sec - exiting 08:50:28 (9352): No heartbeat from core client for 30 sec - exiting 08:50:29 (9352): No heartbeat from core client for 30 sec - exiting 08:50:30 (9352): No heartbeat from core client for 30 sec - exiting 08:50:31 (9352): No heartbeat from core client for 30 sec - exiting 08:50:32 (9352): No heartbeat from core client for 30 sec - exiting 08:50:33 (9352): No heartbeat from core client for 30 sec - exiting 08:50:34 (9352): No heartbeat from core client for 30 sec - exiting 08:50:35 (9352): No heartbeat from core client for 30 sec - exiting 08:50:36 (9352): No heartbeat from core client for 30 sec - exiting 08:50:37 (9352): No heartbeat from core client for 30 sec - exiting 08:50:38 (9352): No heartbeat from core client for 30 sec - exiting 08:50:39 (9352): No heartbeat from core client for 30 sec - exiting 08:50:40 (9352): No heartbeat from core client for 30 sec - exiting 08:50:41 (9352): No heartbeat from core client for 30 sec - exiting 08:50:42 (9352): No heartbeat from core client for 30 sec - exiting 08:50:43 (9352): No heartbeat from core client for 30 sec - exiting 08:50:44 (9352): No heartbeat from core client for 30 sec - exiting 08:50:45 (9352): No heartbeat from core client for 30 sec - exiting 08:50:46 (9352): No heartbeat from core client for 30 sec - exiting 08:50:47 (9352): No heartbeat from core client for 30 sec - exiting 08:50:48 (9352): No heartbeat from core client for 30 sec - exiting 08:50:49 (9352): No heartbeat from core client for 30 sec - exiting 08:50:50 (9352): No heartbeat from core client for 30 sec - exiting 08:50:51 (9352): No heartbeat from core client for 30 sec - exiting 08:50:52 (9352): No heartbeat from core client for 30 sec - exiting 08:50:53 (9352): No heartbeat from core client for 30 sec - exiting 08:50:54 (9352): No heartbeat from core client for 30 sec - exiting 08:50:55 (9352): No heartbeat from core client for 30 sec - exiting 08:50:56 (9352): No heartbeat from core client for 30 sec - exiting 08:50:57 (9352): No heartbeat from core client for 30 sec - exiting 08:50:58 (9352): No heartbeat from core client for 30 sec - exiting 08:50:59 (9352): No heartbeat from core client for 30 sec - exiting 08:51:00 (9352): No heartbeat from core client for 30 sec - exiting 08:51:01 (9352): No heartbeat from core client for 30 sec - exiting 08:51:02 (9352): No heartbeat from core client for 30 sec - exiting 08:51:03 (9352): No heartbeat from core client for 30 sec - exiting 08:51:04 (9352): No heartbeat from core client for 30 sec - exiting 08:51:05 (9352): No heartbeat from core client for 30 sec - exiting 08:51:06 (9352): No heartbeat from core client for 30 sec - exiting 08:51:07 (9352): No heartbeat from core client for 30 sec - exiting 08:51:08 (9352): No heartbeat from core client for 30 sec - exiting 08:51:09 (9352): No heartbeat from core client for 30 sec - exiting 08:51:10 (9352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:51:11 (9352): No heartbeat from core client for 30 sec - exiting 08:51:12 (9352): No heartbeat from core client for 30 sec - exiting 08:51:13 (9352): No heartbeat from core client for 30 sec - exiting 08:51:14 (9352): No heartbeat from core client for 30 sec - exiting 08:55:40 (9784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:41 (9784): No heartbeat from core client for 30 sec - exiting 08:55:42 (9784): No heartbeat from core client for 30 sec - exiting 08:55:43 (9784): No heartbeat from core client for 30 sec - exiting 08:55:44 (9784): No heartbeat from core client for 30 sec - exiting 09:04:20 (7356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:38 (9940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:39 (9940): No heartbeat from core client for 30 sec - exiting 09:13:40 (9940): No heartbeat from core client for 30 sec - exiting 09:13:41 (9940): No heartbeat from core client for 30 sec - exiting 09:19:51 (1268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:26:21 (1708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:14 (9280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:15 (9280): No heartbeat from core client for 30 sec - exiting 09:39:16 (9280): No heartbeat from core client for 30 sec - exiting 09:39:17 (9280): No heartbeat from core client for 30 sec - exiting 09:43:11 (10128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:12 (10128): No heartbeat from core client for 30 sec - exiting 09:43:13 (10128): No heartbeat from core client for 30 sec - exiting 09:46:19 (8832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:40 (9272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:41 (9272): No heartbeat from core client for 30 sec - exiting 10:04:55 (10868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:08:13 (10696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:14:55 (10460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:14:56 (10460): No heartbeat from core client for 30 sec - exiting 10:14:57 (10460): No heartbeat from core client for 30 sec - exiting 10:21:11 (8548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:19 (10356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:20 (10356): No heartbeat from core client for 30 sec - exiting 10:24:21 (10356): No heartbeat from core client for 30 sec - exiting 10:24:22 (10356): No heartbeat from core client for 30 sec - exiting 10:30:29 (2196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:30 (2196): No heartbeat from core client for 30 sec - exiting 10:30:31 (2196): No heartbeat from core client for 30 sec - exiting 10:30:32 (2196): No heartbeat from core client for 30 sec - exiting 10:36:38 (9912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:36:39 (9912): No heartbeat from core client for 30 sec - exiting 10:42:43 (9372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:53 (8260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:54 (8260): No heartbeat from core client for 30 sec - exiting 10:45:55 (8260): No heartbeat from core client for 30 sec - exiting 10:45:56 (8260): No heartbeat from core client for 30 sec - exiting 10:45:57 (8260): No heartbeat from core client for 30 sec - exiting 10:51:57 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:01:05 (10396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:01:06 (10396): No heartbeat from core client for 30 sec - exiting 11:01:07 (10396): No heartbeat from core client for 30 sec - exiting 11:01:08 (10396): No heartbeat from core client for 30 sec - exiting 11:07:22 (10808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:07:23 (10808): No heartbeat from core client for 30 sec - exiting 11:07:24 (10808): No heartbeat from core client for 30 sec - exiting 11:07:25 (10808): No heartbeat from core client for 30 sec - exiting 11:10:29 (11012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:10:30 (11012): No heartbeat from core client for 30 sec - exiting 11:10:31 (11012): No heartbeat from core client for 30 sec - exiting 11:10:32 (11012): No heartbeat from core client for 30 sec - exiting 11:10:33 (11012): No heartbeat from core client for 30 sec - exiting 11:10:34 (11012): No heartbeat from core client for 30 sec - exiting 11:10:35 (11012): No heartbeat from core client for 30 sec - exiting 11:13:43 (7132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:44 (7132): No heartbeat from core client for 30 sec - exiting 11:13:45 (7132): No heartbeat from core client for 30 sec - exiting 11:19:54 (9092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:11 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:12 (6048): No heartbeat from core client for 30 sec - exiting 11:23:13 (6048): No heartbeat from core client for 30 sec - exiting 11:29:15 (10300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:29:16 (10300): No heartbeat from core client for 30 sec - exiting 11:29:17 (10300): No heartbeat from core client for 30 sec - exiting 11:29:18 (10300): No heartbeat from core client for 30 sec - exiting 11:29:19 (10300): No heartbeat from core client for 30 sec - exiting 11:32:22 (8224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:41:36 (9668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:41:37 (9668): No heartbeat from core client for 30 sec - exiting 11:50:37 (10916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:59 (10308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:03:00 (10308): No heartbeat from core client for 30 sec - exiting 12:03:01 (10308): No heartbeat from core client for 30 sec - exiting 12:09:01 (10868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:09:02 (10868): No heartbeat from core client for 30 sec - exiting 12:09:03 (10868): No heartbeat from core client for 30 sec - exiting 12:09:04 (10868): No heartbeat from core client for 30 sec - exiting 12:13:20 (7348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:41 (9560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:42 (9560): No heartbeat from core client for 30 sec - exiting 12:28:43 (9560): No heartbeat from core client for 30 sec - exiting 12:28:44 (9560): No heartbeat from core client for 30 sec - exiting 12:43:46 (11036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:43:47 (11036): No heartbeat from core client for 30 sec - exiting 12:43:48 (11036): No heartbeat from core client for 30 sec - exiting 12:43:49 (11036): No heartbeat from core client for 30 sec - exiting 12:43:50 (11036): No heartbeat from core client for 30 sec - exiting 12:43:51 (11036): No heartbeat from core client for 30 sec - exiting 12:55:55 (11192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:55:56 (11192): No heartbeat from core client for 30 sec - exiting 12:55:57 (11192): No heartbeat from core client for 30 sec - exiting 12:55:58 (11192): No heartbeat from core client for 30 sec - exiting 12:55:59 (11192): No heartbeat from core client for 30 sec - exiting 12:59:05 (10620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:05:14 (12236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:05:15 (12236): No heartbeat from core client for 30 sec - exiting 13:05:16 (12236): No heartbeat from core client for 30 sec - exiting 13:14:48 (12280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:14:49 (12280): No heartbeat from core client for 30 sec - exiting 13:14:50 (12280): No heartbeat from core client for 30 sec - exiting 13:14:51 (12280): No heartbeat from core client for 30 sec - exiting 13:14:52 (12280): No heartbeat from core client for 30 sec - exiting 13:17:56 (12072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:17:57 (12072): No heartbeat from core client for 30 sec - exiting 13:17:58 (12072): No heartbeat from core client for 30 sec - exiting 13:21:04 (4616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:05 (4616): No heartbeat from core client for 30 sec - exiting 13:27:19 (10548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:30:28 (11952): No heartbeat from core client for 30 sec - exiting 13:30:29 (11952): No heartbeat from core client for 30 sec - exiting 13:30:30 (11952): No heartbeat from core client for 30 sec - exiting 13:30:31 (11952): No heartbeat from core client for 30 sec - exiting 13:30:32 (11952): No heartbeat from core client for 30 sec - exiting 13:30:33 (11952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:30:34 (11952): No heartbeat from core client for 30 sec - exiting 13:30:35 (11952): No heartbeat from core client for 30 sec - exiting 13:30:36 (11952): No heartbeat from core client for 30 sec - exiting 13:30:37 (11952): No heartbeat from core client for 30 sec - exiting 13:33:34 (10904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:33:35 (10904): No heartbeat from core client for 30 sec - exiting 13:44:19 (10660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:20 (10660): No heartbeat from core client for 30 sec - exiting 13:44:21 (10660): No heartbeat from core client for 30 sec - exiting 13:51:20 (11344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:51:21 (11344): No heartbeat from core client for 30 sec - exiting 13:51:22 (11344): No heartbeat from core client for 30 sec - exiting 13:51:23 (11344): No heartbeat from core client for 30 sec - exiting 13:51:24 (11344): No heartbeat from core client for 30 sec - exiting 14:06:34 (9916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:06:35 (9916): No heartbeat from core client for 30 sec - exiting 14:25:25 (12252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:26 (12252): No heartbeat from core client for 30 sec - exiting 14:25:27 (12252): No heartbeat from core client for 30 sec - exiting 14:31:35 (9196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:31:36 (9196): No heartbeat from core client for 30 sec - exiting 14:31:37 (9196): No heartbeat from core client for 30 sec - exiting 14:31:38 (9196): No heartbeat from core client for 30 sec - exiting 14:31:39 (9196): No heartbeat from core client for 30 sec - exiting 14:38:16 (248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:33 (3348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:34 (3348): No heartbeat from core client for 30 sec - exiting 14:44:35 (3348): No heartbeat from core client for 30 sec - exiting 14:44:36 (3348): No heartbeat from core client for 30 sec - exiting 14:48:17 (11748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:57:29 (11752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:39 (10496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:40 (10496): No heartbeat from core client for 30 sec - exiting 15:16:19 (11388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:22:25 (11960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:22:26 (11960): No heartbeat from core client for 30 sec - exiting 15:22:27 (11960): No heartbeat from core client for 30 sec - exiting 15:22:28 (11960): No heartbeat from core client for 30 sec - exiting 15:22:29 (11960): No heartbeat from core client for 30 sec - exiting 15:28:33 (10908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:28:34 (10908): No heartbeat from core client for 30 sec - exiting 15:28:35 (10908): No heartbeat from core client for 30 sec - exiting 15:28:36 (10908): No heartbeat from core client for 30 sec - exiting 15:28:37 (10908): No heartbeat from core client for 30 sec - exiting 15:31:40 (10892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:41 (10892): No heartbeat from core client for 30 sec - exiting 15:31:42 (10892): No heartbeat from core client for 30 sec - exiting 15:40:59 (5464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:41:00 (5464): No heartbeat from core client for 30 sec - exiting 15:41:01 (5464): No heartbeat from core client for 30 sec - exiting 15:44:00 (12260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:44:01 (12260): No heartbeat from core client for 30 sec - exiting 15:44:02 (12260): No heartbeat from core client for 30 sec - exiting 15:44:03 (12260): No heartbeat from core client for 30 sec - exiting 15:44:04 (12260): No heartbeat from core client for 30 sec - exiting 15:44:05 (12260): No heartbeat from core client for 30 sec - exiting 15:48:13 (12276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:14 (12276): No heartbeat from core client for 30 sec - exiting 15:48:15 (12276): No heartbeat from core client for 30 sec - exiting 15:48:16 (12276): No heartbeat from core client for 30 sec - exiting 15:48:17 (12276): No heartbeat from core client for 30 sec - exiting 15:57:34 (2924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:57:35 (2924): No heartbeat from core client for 30 sec - exiting 15:57:36 (2924): No heartbeat from core client for 30 sec - exiting 15:57:37 (2924): No heartbeat from core client for 30 sec - exiting 16:00:35 (11708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:36 (11708): No heartbeat from core client for 30 sec - exiting 16:06:42 (10428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:06:43 (10428): No heartbeat from core client for 30 sec - exiting 16:06:44 (10428): No heartbeat from core client for 30 sec - exiting 16:06:45 (10428): No heartbeat from core client for 30 sec - exiting 16:12:56 (4976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:12:57 (4976): No heartbeat from core client for 30 sec - exiting 16:12:58 (4976): No heartbeat from core client for 30 sec - exiting 16:12:59 (4976): No heartbeat from core client for 30 sec - exiting 16:13:00 (4976): No heartbeat from core client for 30 sec - exiting 16:13:01 (4976): No heartbeat from core client for 30 sec - exiting 16:15:56 (12020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:15:57 (12020): No heartbeat from core client for 30 sec - exiting 16:15:58 (12020): No heartbeat from core client for 30 sec - exiting 16:15:59 (12020): No heartbeat from core client for 30 sec - exiting 16:28:01 (8944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:02 (8944): No heartbeat from core client for 30 sec - exiting 16:28:03 (8944): No heartbeat from core client for 30 sec - exiting 16:28:04 (8944): No heartbeat from core client for 30 sec - exiting 16:28:05 (8944): No heartbeat from core client for 30 sec - exiting 16:34:02 (9504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:03 (9504): No heartbeat from core client for 30 sec - exiting 16:34:04 (9504): No heartbeat from core client for 30 sec - exiting 16:34:05 (9504): No heartbeat from core client for 30 sec - exiting 16:34:06 (9504): No heartbeat from core client for 30 sec - exiting 16:37:13 (12168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:37:14 (12168): No heartbeat from core client for 30 sec - exiting 16:37:15 (12168): No heartbeat from core client for 30 sec - exiting 16:49:24 (9484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:49:25 (9484): No heartbeat from core client for 30 sec - exiting 16:55:27 (10524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:28 (10524): No heartbeat from core client for 30 sec - exiting 16:55:29 (10524): No heartbeat from core client for 30 sec - exiting 16:55:30 (10524): No heartbeat from core client for 30 sec - exiting 16:55:31 (10524): No heartbeat from core client for 30 sec - exiting 16:55:32 (10524): No heartbeat from core client for 30 sec - exiting 17:01:37 (10880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:01:38 (10880): No heartbeat from core client for 30 sec - exiting 17:01:39 (10880): No heartbeat from core client for 30 sec - exiting 17:01:40 (10880): No heartbeat from core client for 30 sec - exiting 17:01:41 (10880): No heartbeat from core client for 30 sec - exiting 17:07:46 (12112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:50 (10752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:51 (10752): No heartbeat from core client for 30 sec - exiting 17:17:00 (8616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:20:05 (11844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:20:06 (11844): No heartbeat from core client for 30 sec - exiting 17:20:07 (11844): No heartbeat from core client for 30 sec - exiting 17:26:22 (8816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:26:23 (8816): No heartbeat from core client for 30 sec - exiting 17:26:24 (8816): No heartbeat from core client for 30 sec - exiting 17:26:25 (8816): No heartbeat from core client for 30 sec - exiting 17:26:26 (8816): No heartbeat from core client for 30 sec - exiting 17:26:27 (8816): No heartbeat from core client for 30 sec - exiting 17:35:41 (9244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:47 (10348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:48 (10348): No heartbeat from core client for 30 sec - exiting 17:47:49 (10348): No heartbeat from core client for 30 sec - exiting 17:47:50 (10348): No heartbeat from core client for 30 sec - exiting 17:47:51 (10348): No heartbeat from core client for 30 sec - exiting 17:47:52 (10348): No heartbeat from core client for 30 sec - exiting 17:47:53 (10348): No heartbeat from core client for 30 sec - exiting 17:47:54 (10348): No heartbeat from core client for 30 sec - exiting 17:47:55 (10348): No heartbeat from core client for 30 sec - exiting 17:53:58 (12188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:53:59 (12188): No heartbeat from core client for 30 sec - exiting 17:54:00 (12188): No heartbeat from core client for 30 sec - exiting 18:00:06 (8832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:00:07 (8832): No heartbeat from core client for 30 sec - exiting 18:03:15 (9736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:17 (12104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:18 (12104): No heartbeat from core client for 30 sec - exiting 18:06:19 (12104): No heartbeat from core client for 30 sec - exiting 18:06:20 (12104): No heartbeat from core client for 30 sec - exiting 18:06:21 (12104): No heartbeat from core client for 30 sec - exiting 18:09:20 (9624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:21:30 (11400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:36 (8224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:37 (8224): No heartbeat from core client for 30 sec - exiting 18:36:42 (11388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:43 (11388): No heartbeat from core client for 30 sec - exiting 18:36:44 (11388): No heartbeat from core client for 30 sec - exiting 18:36:45 (11388): No heartbeat from core client for 30 sec - exiting 18:36:46 (11388): No heartbeat from core client for 30 sec - exiting 18:45:48 (9736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:45:49 (9736): No heartbeat from core client for 30 sec - exiting 18:45:50 (9736): No heartbeat from core client for 30 sec - exiting 18:45:51 (9736): No heartbeat from core client for 30 sec - exiting 18:51:56 (10180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:51:57 (10180): No heartbeat from core client for 30 sec - exiting 18:51:58 (10180): No heartbeat from core client for 30 sec - exiting 18:58:03 (12148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:04 (12148): No heartbeat from core client for 30 sec - exiting 18:58:05 (12148): No heartbeat from core client for 30 sec - exiting 18:58:06 (12148): No heartbeat from core client for 30 sec - exiting 18:58:07 (12148): No heartbeat from core client for 30 sec - exiting 19:07:10 (10756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:07:11 (10756): No heartbeat from core client for 30 sec - exiting 19:07:12 (10756): No heartbeat from core client for 30 sec - exiting 19:07:13 (10756): No heartbeat from core client for 30 sec - exiting 19:07:14 (10756): No heartbeat from core client for 30 sec - exiting 19:07:15 (10756): No heartbeat from core client for 30 sec - exiting 19:13:12 (8832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:13 (8832): No heartbeat from core client for 30 sec - exiting 19:13:14 (8832): No heartbeat from core client for 30 sec - exiting 19:13:15 (8832): No heartbeat from core client for 30 sec - exiting 19:13:16 (8832): No heartbeat from core client for 30 sec - exiting 19:13:17 (8832): No heartbeat from core client for 30 sec - exiting 19:19:16 (11600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:17 (11600): No heartbeat from core client for 30 sec - exiting 19:28:23 (9744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:28:24 (9744): No heartbeat from core client for 30 sec - exiting 19:28:25 (9744): No heartbeat from core client for 30 sec - exiting 19:28:26 (9744): No heartbeat from core client for 30 sec - exiting 19:28:27 (9744): No heartbeat from core client for 30 sec - exiting 19:34:27 (11600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:28 (11600): No heartbeat from core client for 30 sec - exiting 19:34:29 (11600): No heartbeat from core client for 30 sec - exiting 19:40:28 (11984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:40:29 (11984): No heartbeat from core client for 30 sec - exiting 19:43:32 (10712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:35 (6968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:36 (6968): No heartbeat from core client for 30 sec - exiting 19:46:37 (6968): No heartbeat from core client for 30 sec - exiting 19:46:38 (6968): No heartbeat from core client for 30 sec - exiting 19:46:39 (6968): No heartbeat from core client for 30 sec - exiting 19:46:40 (6968): No heartbeat from core client for 30 sec - exiting 19:46:41 (6968): No heartbeat from core client for 30 sec - exiting 19:52:47 (11216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:52:48 (11216): No heartbeat from core client for 30 sec - exiting 19:52:49 (11216): No heartbeat from core client for 30 sec - exiting 19:52:50 (11216): No heartbeat from core client for 30 sec - exiting 19:52:51 (11216): No heartbeat from core client for 30 sec - exiting 19:58:54 (11132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Jun 2013 07:42:57	1189257	15832067	hadcm3n_o09k_1980_40_008388707_1	77,760	166,455	2.1406
21 Jun 2013 12:25:04	1189257	15832067	hadcm3n_o09k_1980_40_008388707_1	51,840	111,081	2.1428
18 Jun 2013 16:31:03	1189257	15832067	hadcm3n_o09k_1980_40_008388707_1	25,920	54,977	2.1210