Task 15553085

Name	hadcm3n_39vf_1980_40_008283521_1
Workunit	8434656
Created	20 Jan 2013, 18:22:33 UTC
Sent	20 Jan 2013, 18:22:41 UTC
Report deadline	22 Apr 2013, 1:49:52 UTC
Received	22 Apr 2013, 14:20:59 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID	1164908
Run time	18 days 4 hours 41 min 50 sec
CPU time	17 days 7 hours 20 min 34 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	2.57 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> 15:58:21 (7680): No heartbeat from core client for 30 sec - exiting 15:58:22 (7680): No heartbeat from core client for 30 sec - exiting 15:58:23 (7680): No heartbeat from core client for 30 sec - exiting 15:58:24 (7680): No heartbeat from core client for 30 sec - exiting 15:58:25 (7680): No heartbeat from core client for 30 sec - exiting 15:58:26 (7680): No heartbeat from core client for 30 sec - exiting 15:58:27 (7680): No heartbeat from core client for 30 sec - exiting 15:58:28 (7680): No heartbeat from core client for 30 sec - exiting 15:58:29 (7680): No heartbeat from core client for 30 sec - exiting 15:58:30 (7680): No heartbeat from core client for 30 sec - exiting 15:58:31 (7680): No heartbeat from core client for 30 sec - exiting 15:58:32 (7680): No heartbeat from core client for 30 sec - exiting 15:58:33 (7680): No heartbeat from core client for 30 sec - exiting 15:58:34 (7680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:35 (5636): No heartbeat from core client for 30 sec - exiting 10:29:36 (5636): No heartbeat from core client for 30 sec - exiting 10:29:37 (5636): No heartbeat from core client for 30 sec - exiting 10:29:38 (5636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:05:13 (5560): No heartbeat from core client for 30 sec - exiting 15:05:14 (5560): No heartbeat from core client for 30 sec - exiting 15:05:15 (5560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:11:32 (6032): No heartbeat from core client for 30 sec - exiting 16:11:35 (6032): No heartbeat from core client for 30 sec - exiting 16:11:36 (6032): No heartbeat from core client for 30 sec - exiting 16:11:37 (6032): No heartbeat from core client for 30 sec - exiting 16:11:38 (6032): No heartbeat from core client for 30 sec - exiting 16:11:39 (6032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:19 (6160): No heartbeat from core client for 30 sec - exiting 14:49:20 (6160): No heartbeat from core client for 30 sec - exiting 14:49:21 (6160): No heartbeat from core client for 30 sec - exiting 14:49:22 (6160): No heartbeat from core client for 30 sec - exiting 14:49:23 (6160): No heartbeat from core client for 30 sec - exiting 14:49:24 (6160): No heartbeat from core client for 30 sec - exiting 14:49:25 (6160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:26 (6160): No heartbeat from core client for 30 sec - exiting 10:34:36 (5920): No heartbeat from core client for 30 sec - exiting 10:34:37 (5920): No heartbeat from core client for 30 sec - exiting 10:34:38 (5920): No heartbeat from core client for 30 sec - exiting 10:34:39 (5920): No heartbeat from core client for 30 sec - exiting 10:34:40 (5920): No heartbeat from core client for 30 sec - exiting 10:34:41 (5920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:53:19 (6140): No heartbeat from core client for 30 sec - exiting 10:53:20 (6140): No heartbeat from core client for 30 sec - exiting 10:53:21 (6140): No heartbeat from core client for 30 sec - exiting 10:53:22 (6140): No heartbeat from core client for 30 sec - exiting 10:53:23 (6140): No heartbeat from core client for 30 sec - exiting 10:53:24 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:48:53 (6272): No heartbeat from core client for 30 sec - exiting 11:48:54 (6272): No heartbeat from core client for 30 sec - exiting 11:48:55 (6272): No heartbeat from core client for 30 sec - exiting 11:48:56 (6272): No heartbeat from core client for 30 sec - exiting 11:48:57 (6272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:02 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7596, iMonCtr=1 Model crash detected, will try to restart... 15:03:12 (5988): No heartbeat from core client for 30 sec - exiting 15:03:13 (5988): No heartbeat from core client for 30 sec - exiting 15:03:14 (5988): No heartbeat from core client for 30 sec - exiting 15:03:15 (5988): No heartbeat from core client for 30 sec - exiting 15:03:16 (5988): No heartbeat from core client for 30 sec - exiting 15:03:17 (5988): No heartbeat from core client for 30 sec - exiting 15:03:18 (5988): No heartbeat from core client for 30 sec - exiting 15:03:19 (5988): No heartbeat from core client for 30 sec - exiting 15:03:20 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:03:21 (5988): No heartbeat from core client for 30 sec - exiting 15:02:04 (5980): No heartbeat from core client for 30 sec - exiting 15:02:05 (5980): No heartbeat from core client for 30 sec - exiting 15:02:06 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5128, iMonCtr=1 Model crash detected, will try to restart... 12:55:54 (5320): No heartbeat from core client for 30 sec - exiting 12:55:56 (5320): No heartbeat from core client for 30 sec - exiting 12:55:57 (5320): No heartbeat from core client for 30 sec - exiting 12:55:58 (5320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:55:59 (5320): No heartbeat from core client for 30 sec - exiting 10:33:08 (5696): No heartbeat from core client for 30 sec - exiting 10:33:09 (5696): No heartbeat from core client for 30 sec - exiting 10:33:10 (5696): No heartbeat from core client for 30 sec - exiting 10:33:11 (5696): No heartbeat from core client for 30 sec - exiting 10:33:12 (5696): No heartbeat from core client for 30 sec - exiting 10:33:13 (5696): No heartbeat from core client for 30 sec - exiting 10:33:14 (5696): No heartbeat from core client for 30 sec - exiting 10:33:15 (5696): No heartbeat from core client for 30 sec - exiting 10:33:16 (5696): No heartbeat from core client for 30 sec - exiting 10:33:17 (5696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:48 (4976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... 10:46:00 (6088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:25 (6052): No heartbeat from core client for 30 sec - exiting 10:30:26 (6052): No heartbeat from core client for 30 sec - exiting 10:30:27 (6052): No heartbeat from core client for 30 sec - exiting 10:30:28 (6052): No heartbeat from core client for 30 sec - exiting 10:30:29 (6052): No heartbeat from core client for 30 sec - exiting 10:30:30 (6052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:14 (6280): No heartbeat from core client for 30 sec - exiting 10:45:15 (6280): No heartbeat from core client for 30 sec - exiting 10:45:16 (6280): No heartbeat from core client for 30 sec - exiting 10:45:17 (6280): No heartbeat from core client for 30 sec - exiting 10:45:18 (6280): No heartbeat from core client for 30 sec - exiting 10:45:19 (6280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:24 (4952): No heartbeat from core client for 30 sec - exiting 14:28:25 (4952): No heartbeat from core client for 30 sec - exiting 14:28:26 (4952): No heartbeat from core client for 30 sec - exiting 14:28:27 (4952): No heartbeat from core client for 30 sec - exiting 14:28:28 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:21:00 (5636): No heartbeat from core client for 30 sec - exiting 11:21:02 (5636): No heartbeat from core client for 30 sec - exiting 11:21:03 (5636): No heartbeat from core client for 30 sec - exiting 11:21:04 (5636): No heartbeat from core client for 30 sec - exiting 11:21:05 (5636): No heartbeat from core client for 30 sec - exiting 11:21:06 (5636): No heartbeat from core client for 30 sec - exiting 11:21:07 (5636): No heartbeat from core client for 30 sec - exiting 11:21:08 (5636): No heartbeat from core client for 30 sec - exiting 11:21:09 (5636): No heartbeat from core client for 30 sec - exiting 11:21:10 (5636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:34 (5660): No heartbeat from core client for 30 sec - exiting 11:23:35 (5660): No heartbeat from core client for 30 sec - exiting 11:23:36 (5660): No heartbeat from core client for 30 sec - exiting 11:23:37 (5660): No heartbeat from core client for 30 sec - exiting 11:23:38 (5660): No heartbeat from core client for 30 sec - exiting 11:23:39 (5660): No heartbeat from core client for 30 sec - exiting 11:23:40 (5660): No heartbeat from core client for 30 sec - exiting 11:23:41 (5660): No heartbeat from core client for 30 sec - exiting 11:23:42 (5660): No heartbeat from core client for 30 sec - exiting 11:23:43 (5660): No heartbeat from core client for 30 sec - exiting 11:23:44 (5660): No heartbeat from core client for 30 sec - exiting 11:23:45 (5660): No heartbeat from core client for 30 sec - exiting 11:23:46 (5660): No heartbeat from core client for 30 sec - exiting 11:23:47 (5660): No heartbeat from core client for 30 sec - exiting 11:23:48 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:07:46 (3312): No heartbeat from core client for 30 sec - exiting 12:07:47 (3312): No heartbeat from core client for 30 sec - exiting 12:07:48 (3312): No heartbeat from core client for 30 sec - exiting 12:07:49 (3312): No heartbeat from core client for 30 sec - exiting 12:07:50 (3312): No heartbeat from core client for 30 sec - exiting 12:07:51 (3312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:48 (6080): No heartbeat from core client for 30 sec - exiting 11:13:49 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:35 (6000): No heartbeat from core client for 30 sec - exiting 11:15:36 (6000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:41:12 (7900): No heartbeat from core client for 30 sec - exiting 14:41:13 (7900): No heartbeat from core client for 30 sec - exiting 14:41:14 (7900): No heartbeat from core client for 30 sec - exiting 14:41:15 (7900): No heartbeat from core client for 30 sec - exiting 14:41:16 (7900): No heartbeat from core client for 30 sec - exiting 14:41:17 (7900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:51 (1680): No heartbeat from core client for 30 sec - exiting 15:37:52 (1680): No heartbeat from core client for 30 sec - exiting 15:37:53 (1680): No heartbeat from core client for 30 sec - exiting 15:37:54 (1680): No heartbeat from core client for 30 sec - exiting 15:37:55 (1680): No heartbeat from core client for 30 sec - exiting 15:37:56 (1680): No heartbeat from core client for 30 sec - exiting 15:37:57 (1680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:39 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:20:21 (1568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7324, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=228, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=228, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=228, iMonCtr=1 Model crash detected, will try to restart... 12:53:47 (6168): No heartbeat from core client for 30 sec - exiting 12:53:48 (6168): No heartbeat from core client for 30 sec - exiting 12:53:49 (6168): No heartbeat from core client for 30 sec - exiting 12:53:50 (6168): No heartbeat from core client for 30 sec - exiting 12:53:51 (6168): No heartbeat from core client for 30 sec - exiting 12:53:52 (6168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:15:08 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:13:29 (5880): No heartbeat from core client for 30 sec - exiting 12:13:30 (5880): No heartbeat from core client for 30 sec - exiting 12:13:31 (5880): No heartbeat from core client for 30 sec - exiting 12:13:32 (5880): No heartbeat from core client for 30 sec - exiting 12:13:33 (5880): No heartbeat from core client for 30 sec - exiting 12:13:34 (5880): No heartbeat from core client for 30 sec - exiting 12:13:35 (5880): No heartbeat from core client for 30 sec - exiting 12:13:36 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:12 (5784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:42 (7256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:03 (6080): No heartbeat from core client for 30 sec - exiting 16:04:04 (6080): No heartbeat from core client for 30 sec - exiting 16:04:05 (6080): No heartbeat from core client for 30 sec - exiting 16:04:06 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:19:04 (5568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:25 (6352): No heartbeat from core client for 30 sec - exiting 14:17:26 (6352): No heartbeat from core client for 30 sec - exiting 14:17:27 (6352): No heartbeat from core client for 30 sec - exiting 14:17:28 (6352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:39 (5476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:41:45 (5596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:29 (6924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:15:45 (4416): No heartbeat from core client for 30 sec - exiting 10:15:47 (4416): No heartbeat from core client for 30 sec - exiting 10:15:48 (4416): No heartbeat from core client for 30 sec - exiting 10:15:49 (4416): No heartbeat from core client for 30 sec - exiting 10:15:50 (4416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:06:38 (4620): No heartbeat from core client for 30 sec - exiting 10:06:40 (4620): No heartbeat from core client for 30 sec - exiting 10:06:41 (4620): No heartbeat from core client for 30 sec - exiting 10:06:42 (4620): No heartbeat from core client for 30 sec - exiting 10:06:43 (4620): No heartbeat from core client for 30 sec - exiting 10:06:44 (4620): No heartbeat from core client for 30 sec - exiting 10:06:45 (4620): No heartbeat from core client for 30 sec - exiting 10:06:46 (4620): No heartbeat from core client for 30 sec - exiting 10:06:47 (4620): No heartbeat from core client for 30 sec - exiting 10:06:48 (4620): No heartbeat from core client for 30 sec - exiting 10:06:49 (4620): No heartbeat from core client for 30 sec - exiting 10:06:50 (4620): No heartbeat from core client for 30 sec - exiting 10:06:51 (4620): No heartbeat from core client for 30 sec - exiting 10:06:52 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:00:18 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:42:02 (6748): No heartbeat from core client for 30 sec - exiting 11:42:03 (6748): No heartbeat from core client for 30 sec - exiting 11:42:04 (6748): No heartbeat from core client for 30 sec - exiting 11:42:05 (6748): No heartbeat from core client for 30 sec - exiting 11:42:06 (6748): No heartbeat from core client for 30 sec - exiting 11:42:07 (6748): No heartbeat from core client for 30 sec - exiting 11:42:08 (6748): No heartbeat from core client for 30 sec - exiting 11:42:09 (6748): No heartbeat from core client for 30 sec - exiting 11:42:10 (6748): No heartbeat from core client for 30 sec - exiting 11:42:11 (6748): No heartbeat from core client for 30 sec - exiting 11:42:12 (6748): No heartbeat from core client for 30 sec - exiting 11:42:13 (6748): No heartbeat from core client for 30 sec - exiting 11:42:14 (6748): No heartbeat from core client for 30 sec - exiting 11:42:15 (6748): No heartbeat from core client for 30 sec - exiting 11:42:16 (6748): No heartbeat from core client for 30 sec - exiting 11:42:17 (6748): No heartbeat from core client for 30 sec - exiting 11:42:18 (6748): No heartbeat from core client for 30 sec - exiting 11:42:19 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:57:31 (6064): No heartbeat from core client for 30 sec - exiting 10:57:32 (6064): No heartbeat from core client for 30 sec - exiting 10:57:33 (6064): No heartbeat from core client for 30 sec - exiting 10:57:34 (6064): No heartbeat from core client for 30 sec - exiting 10:57:35 (6064): No heartbeat from core client for 30 sec - exiting 10:57:36 (6064): No heartbeat from core client for 30 sec - exiting 10:57:37 (6064): No heartbeat from core client for 30 sec - exiting 10:57:38 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:55 (6116): No heartbeat from core client for 30 sec - exiting 10:30:57 (6116): No heartbeat from core client for 30 sec - exiting 10:30:58 (6116): No heartbeat from core client for 30 sec - exiting 10:30:59 (6116): No heartbeat from core client for 30 sec - exiting 10:31:00 (6116): No heartbeat from core client for 30 sec - exiting 10:31:01 (6116): No heartbeat from core client for 30 sec - exiting 10:31:02 (6116): No heartbeat from core client for 30 sec - exiting 10:31:03 (6116): No heartbeat from core client for 30 sec - exiting 10:31:04 (6116): No heartbeat from core client for 30 sec - exiting 10:31:05 (6116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:53 (8452): No heartbeat from core client for 30 sec - exiting 18:52:54 (8452): No heartbeat from core client for 30 sec - exiting 18:52:55 (8452): No heartbeat from core client for 30 sec - exiting 18:52:56 (8452): No heartbeat from core client for 30 sec - exiting 18:52:57 (8452): No heartbeat from core client for 30 sec - exiting 18:52:58 (8452): No heartbeat from core client for 30 sec - exiting 18:52:59 (8452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9064, iMonCtr=1 Model crash detected, will try to restart... 11:12:26 (6048): No heartbeat from core client for 30 sec - exiting 11:12:28 (6048): No heartbeat from core client for 30 sec - exiting 11:12:29 (6048): No heartbeat from core client for 30 sec - exiting 11:12:30 (6048): No heartbeat from core client for 30 sec - exiting 11:12:31 (6048): No heartbeat from core client for 30 sec - exiting 11:12:32 (6048): No heartbeat from core client for 30 sec - exiting 11:12:33 (6048): No heartbeat from core client for 30 sec - exiting 11:12:34 (6048): No heartbeat from core client for 30 sec - exiting 11:12:35 (6048): No heartbeat from core client for 30 sec - exiting 11:12:36 (6048): No heartbeat from core client for 30 sec - exiting 11:12:37 (6048): No heartbeat from core client for 30 sec - exiting 11:12:38 (6048): No heartbeat from core client for 30 sec - exiting 11:12:39 (6048): No heartbeat from core client for 30 sec - exiting 11:12:40 (6048): No heartbeat from core client for 30 sec - exiting 11:12:41 (6048): No heartbeat from core client for 30 sec - exiting 11:12:42 (6048): No heartbeat from core client for 30 sec - exiting 11:12:43 (6048): No heartbeat from core client for 30 sec - exiting 11:12:44 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:43:54 (6168): No heartbeat from core client for 30 sec - exiting 13:43:55 (6168): No heartbeat from core client for 30 sec - exiting 13:43:56 (6168): No heartbeat from core client for 30 sec - exiting 13:43:57 (6168): No heartbeat from core client for 30 sec - exiting 13:43:58 (6168): No heartbeat from core client for 30 sec - exiting 13:43:59 (6168): No heartbeat from core client for 30 sec - exiting 13:44:00 (6168): No heartbeat from core client for 30 sec - exiting 13:44:01 (6168): No heartbeat from core client for 30 sec - exiting 13:44:02 (6168): No heartbeat from core client for 30 sec - exiting 13:44:03 (6168): No heartbeat from core client for 30 sec - exiting 13:44:04 (6168): No heartbeat from core client for 30 sec - exiting 13:44:05 (6168): No heartbeat from core client for 30 sec - exiting 13:44:06 (6168): No heartbeat from core client for 30 sec - exiting 13:44:07 (6168): No heartbeat from core client for 30 sec - exiting 13:44:08 (6168): No heartbeat from core client for 30 sec - exiting 13:44:09 (6168): No heartbeat from core client for 30 sec - exiting 13:44:10 (6168): No heartbeat from core client for 30 sec - exiting 13:44:11 (6168): No heartbeat from core client for 30 sec - exiting 13:44:12 (6168): No heartbeat from core client for 30 sec - exiting 13:44:13 (6168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:12 (5404): No heartbeat from core client for 30 sec - exiting 15:09:13 (5404): No heartbeat from core client for 30 sec - exiting 15:09:14 (5404): No heartbeat from core client for 30 sec - exiting 15:09:15 (5404): No heartbeat from core client for 30 sec - exiting 15:09:16 (5404): No heartbeat from core client for 30 sec - exiting 15:09:17 (5404): No heartbeat from core client for 30 sec - exiting 15:09:18 (5404): No heartbeat from core client for 30 sec - exiting 15:09:19 (5404): No heartbeat from core client for 30 sec - exiting 15:09:20 (5404): No heartbeat from core client for 30 sec - exiting 15:09:21 (5404): No heartbeat from core client for 30 sec - exiting 15:09:22 (5404): No heartbeat from core client for 30 sec - exiting 15:09:23 (5404): No heartbeat from core client for 30 sec - exiting 15:09:24 (5404): No heartbeat from core client for 30 sec - exiting 15:09:25 (5404): No heartbeat from core client for 30 sec - exiting 15:09:26 (5404): No heartbeat from core client for 30 sec - exiting 15:09:27 (5404): No heartbeat from core client for 30 sec - exiting 15:09:28 (5404): No heartbeat from core client for 30 sec - exiting 15:09:29 (5404): No heartbeat from core client for 30 sec - exiting 15:09:30 (5404): No heartbeat from core client for 30 sec - exiting 15:09:31 (5404): No heartbeat from core client for 30 sec - exiting 15:09:32 (5404): No heartbeat from core client for 30 sec - exiting 15:09:33 (5404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:26:28 (3908): No heartbeat from core client for 30 sec - exiting 15:26:30 (3908): No heartbeat from core client for 30 sec - exiting 15:26:31 (3908): No heartbeat from core client for 30 sec - exiting 15:26:32 (3908): No heartbeat from core client for 30 sec - exiting 15:26:33 (3908): No heartbeat from core client for 30 sec - exiting 15:26:34 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6192, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish 15:21:10 (8048): No heartbeat from core client for 30 sec - exiting 15:21:11 (8048): No heartbeat from core client for 30 sec - exiting 15:21:12 (8048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:24:33 (4744): No heartbeat from core client for 30 sec - exiting 15:24:34 (4744): No heartbeat from core client for 30 sec - exiting 15:24:35 (4744): No heartbeat from core client for 30 sec - exiting 15:24:36 (4744): No heartbeat from core client for 30 sec - exiting 15:24:37 (4744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:29:32 (5752): No heartbeat from core client for 30 sec - exiting 09:29:33 (5752): No heartbeat from core client for 30 sec - exiting 09:29:34 (5752): No heartbeat from core client for 30 sec - exiting 09:29:35 (5752): No heartbeat from core client for 30 sec - exiting 09:29:36 (5752): No heartbeat from core client for 30 sec - exiting 09:29:37 (5752): No heartbeat from core client for 30 sec - exiting 09:29:38 (5752): No heartbeat from core client for 30 sec - exiting 09:29:39 (5752): No heartbeat from core client for 30 sec - exiting 09:29:40 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:05:45 (6260): No heartbeat from core client for 30 sec - exiting 10:05:47 (6260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Abort request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Apr 2013 15:25:39	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	829,440	1,416,777	1.7081
13 Apr 2013 15:20:49	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	803,520	1,373,583	1.7095
12 Apr 2013 13:40:07	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	777,600	1,329,075	1.7092
06 Apr 2013 20:07:34	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	751,680	1,286,840	1.7120
05 Apr 2013 18:48:35	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	725,760	1,244,043	1.7141
04 Apr 2013 18:17:38	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	699,840	1,200,557	1.7155
03 Apr 2013 15:05:38	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	673,920	1,153,741	1.7120
01 Apr 2013 15:30:49	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	648,000	1,104,777	1.7049
31 Mar 2013 13:10:35	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	622,080	1,056,702	1.6987
30 Mar 2013 11:35:19	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	596,160	1,007,961	1.6908
28 Mar 2013 20:24:24	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	570,240	959,702	1.6830
24 Mar 2013 13:55:08	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	544,320	916,482	1.6837
22 Mar 2013 21:06:36	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	518,400	875,649	1.6891
17 Mar 2013 13:50:17	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	492,480	833,262	1.6920
16 Mar 2013 10:23:43	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	466,560	788,577	1.6902
10 Mar 2013 15:55:38	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	440,640	744,171	1.6888
09 Mar 2013 10:22:43	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	414,720	698,414	1.6841
03 Mar 2013 16:00:43	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	388,800	654,261	1.6828
02 Mar 2013 14:35:30	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	362,880	610,478	1.6823
24 Feb 2013 19:44:10	1164908	15553085	hadcm3n_39vf_1980_40_008283521_1	336,960	564,602	1.6756