Task 16052215

Name	hadcm3n_ocyt_1900_40_008471928_2
Workunit	8622767
Created	1 Oct 2013, 14:04:19 UTC
Sent	1 Oct 2013, 14:08:34 UTC
Report deadline	31 Dec 2013, 21:35:45 UTC
Received	10 Dec 2013, 21:34:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1202025
Run time	20 days 14 hours 47 min 35 sec
CPU time	18 days 19 hours 44 min 56 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 20:36:56 (5480): No heartbeat from core client for 30 sec - exiting 20:36:57 (5480): No heartbeat from core client for 30 sec - exiting 20:36:58 (5480): No heartbeat from core client for 30 sec - exiting 20:36:59 (5480): No heartbeat from core client for 30 sec - exiting 20:37:00 (5480): No heartbeat from core client for 30 sec - exiting 20:37:01 (5480): No heartbeat from core client for 30 sec - exiting 20:37:03 (5480): No heartbeat from core client for 30 sec - exiting 20:37:04 (5480): No heartbeat from core client for 30 sec - exiting 20:37:05 (5480): No heartbeat from core client for 30 sec - exiting 20:37:06 (5480): No heartbeat from core client for 30 sec - exiting 20:37:07 (5480): No heartbeat from core client for 30 sec - exiting 20:37:08 (5480): No heartbeat from core client for 30 sec - exiting 20:37:09 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:03:38 (3160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6096, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6096, iMonCtr=1 Model crash detected, will try to restart... C19:33:03 (5808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:33:27 (5416): No heartbeat from core client for 30 sec - exiting 22:33:28 (5416): No heartbeat from core client for 30 sec - exiting 22:33:29 (5416): No heartbeat from core client for 30 sec - exiting 22:33:30 (5416): No heartbeat from core client for 30 sec - exiting 22:33:31 (5416): No heartbeat from core client for 30 sec - exiting 22:33:32 (5416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:06:18 (6076): No heartbeat from core client for 30 sec - exiting 22:06:19 (6076): No heartbeat from core client for 30 sec - exiting 22:06:21 (6076): No heartbeat from core client for 30 sec - exiting 22:06:22 (6076): No heartbeat from core client for 30 sec - exiting 22:06:23 (6076): No heartbeat from core client for 30 sec - exiting 22:06:24 (6076): No heartbeat from core client for 30 sec - exiting 22:06:25 (6076): No heartbeat from core client for 30 sec - exiting 22:06:26 (6076): No heartbeat from core client for 30 sec - exiting 22:06:27 (6076): No heartbeat from core client for 30 sec - exiting 22:06:28 (6076): No heartbeat from core client for 30 sec - exiting 22:06:29 (6076): No heartbeat from core client for 30 sec - exiting 22:06:30 (6076): No heartbeat from core client for 30 sec - exiting 22:06:31 (6076): No heartbeat from core client for 30 sec - exiting 22:06:33 (6076): No heartbeat from core client for 30 sec - exiting 22:06:34 (6076): No heartbeat from core client for 30 sec - exiting 22:06:35 (6076): No heartbeat from core client for 30 sec - exiting 22:06:36 (6076): No heartbeat from core client for 30 sec - exiting 22:06:37 (6076): No heartbeat from core client for 30 sec - exiting 22:06:38 (6076): No heartbeat from core client for 30 sec - exiting 22:06:39 (6076): No heartbeat from core client for 30 sec - exiting 22:06:40 (6076): No heartbeat from core client for 30 sec - exiting 22:06:41 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:15:40 (6128): No heartbeat from core client for 30 sec - exiting 20:15:41 (6128): No heartbeat from core client for 30 sec - exiting 20:15:42 (6128): No heartbeat from core client for 30 sec - exiting 20:15:43 (6128): No heartbeat from core client for 30 sec - exiting 20:15:44 (6128): No heartbeat from core client for 30 sec - exiting 20:15:45 (6128): No heartbeat from core client for 30 sec - exiting 20:15:46 (6128): No heartbeat from core client for 30 sec - exiting 20:15:47 (6128): No heartbeat from core client for 30 sec - exiting 20:15:48 (6128): No heartbeat from core client for 30 sec - exiting 20:15:49 (6128): No heartbeat from core client for 30 sec - exiting 20:15:50 (6128): No heartbeat from core client for 30 sec - exiting 20:15:51 (6128): No heartbeat from core client for 30 sec - exiting 20:15:52 (6128): No heartbeat from core client for 30 sec - exiting 20:15:53 (6128): No heartbeat from core client for 30 sec - exiting 20:15:54 (6128): No heartbeat from core client for 30 sec - exiting 20:15:55 (6128): No heartbeat from core client for 30 sec - exiting 20:15:56 (6128): No heartbeat from core client for 30 sec - exiting 20:15:57 (6128): No heartbeat from core client for 30 sec - exiting 20:15:58 (6128): No heartbeat from core client for 30 sec - exiting 20:15:59 (6128): No heartbeat from core client for 30 sec - exiting 20:16:00 (6128): No heartbeat from core client for 30 sec - exiting 20:16:01 (6128): No heartbeat from core client for 30 sec - exiting 20:16:02 (6128): No heartbeat from core client for 30 sec - exiting 20:16:03 (6128): No heartbeat from core client for 30 sec - exiting 20:16:04 (6128): No heartbeat from core client for 30 sec - exiting 20:16:05 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:55:59 (6052): No heartbeat from core client for 30 sec - exiting 20:56:00 (6052): No heartbeat from core client for 30 sec - exiting 20:56:01 (6052): No heartbeat from core client for 30 sec - exiting 20:56:02 (6052): No heartbeat from core client for 30 sec - exiting 20:56:03 (6052): No heartbeat from core client for 30 sec - exiting 20:56:04 (6052): No heartbeat from core client for 30 sec - exiting 20:56:05 (6052): No heartbeat from core client for 30 sec - exiting 20:56:06 (6052): No heartbeat from core client for 30 sec - exiting 20:56:07 (6052): No heartbeat from core client for 30 sec - exiting 20:56:08 (6052): No heartbeat from core client for 30 sec - exiting 20:56:09 (6052): No heartbeat from core client for 30 sec - exiting 20:56:10 (6052): No heartbeat from core client for 30 sec - exiting 20:56:11 (6052): No heartbeat from core client for 30 sec - exiting 20:56:12 (6052): No heartbeat from core client for 30 sec - exiting 20:56:13 (6052): No heartbeat from core client for 30 sec - exiting 20:56:14 (6052): No heartbeat from core client for 30 sec - exiting 20:56:15 (6052): No heartbeat from core client for 30 sec - exiting 20:56:16 (6052): No heartbeat from core client for 30 sec - exiting 20:56:17 (6052): No heartbeat from core client for 30 sec - exiting 20:56:18 (6052): No heartbeat from core client for 30 sec - exiting 20:56:19 (6052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:30:54 (4824): No heartbeat from core client for 30 sec - exiting 19:30:56 (4824): No heartbeat from core client for 30 sec - exiting 19:30:57 (4824): No heartbeat from core client for 30 sec - exiting 19:30:58 (4824): No heartbeat from core client for 30 sec - exiting 19:30:59 (4824): No heartbeat from core client for 30 sec - exiting 19:31:00 (4824): No heartbeat from core client for 30 sec - exiting 19:31:01 (4824): No heartbeat from core client for 30 sec - exiting 19:31:02 (4824): No heartbeat from core client for 30 sec - exiting 19:31:03 (4824): No heartbeat from core client for 30 sec - exiting 19:31:04 (4824): No heartbeat from core client for 30 sec - exiting 19:31:05 (4824): No heartbeat from core client for 30 sec - exiting 19:31:06 (4824): No heartbeat from core client for 30 sec - exiting 19:31:08 (4824): No heartbeat from core client for 30 sec - exiting 19:31:09 (4824): No heartbeat from core client for 30 sec - exiting 19:31:10 (4824): No heartbeat from core client for 30 sec - exiting 19:31:11 (4824): No heartbeat from core client for 30 sec - exiting 19:31:12 (4824): No heartbeat from core client for 30 sec - exiting 19:31:13 (4824): No heartbeat from core client for 30 sec - exiting 19:31:14 (4824): No heartbeat from core client for 30 sec - exiting 19:31:15 (4824): No heartbeat from core client for 30 sec - exiting 19:31:16 (4824): No heartbeat from core client for 30 sec - exiting 19:31:17 (4824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:18 (4824): No heartbeat from core client for 30 sec - exiting 18:27:56 (5612): No heartbeat from core client for 30 sec - exiting 18:27:57 (5612): No heartbeat from core client for 30 sec - exiting 18:27:58 (5612): No heartbeat from core client for 30 sec - exiting 18:27:59 (5612): No heartbeat from core client for 30 sec - exiting 18:28:00 (5612): No heartbeat from core client for 30 sec - exiting 18:28:01 (5612): No heartbeat from core client for 30 sec - exiting 18:28:02 (5612): No heartbeat from core client for 30 sec - exiting 18:28:03 (5612): No heartbeat from core client for 30 sec - exiting 18:28:04 (5612): No heartbeat from core client for 30 sec - exiting 18:28:05 (5612): No heartbeat from core client for 30 sec - exiting 18:28:06 (5612): No heartbeat from core client for 30 sec - exiting 18:28:07 (5612): No heartbeat from core client for 30 sec - exiting 18:28:08 (5612): No heartbeat from core client for 30 sec - exiting 18:28:09 (5612): No heartbeat from core client for 30 sec - exiting 18:28:10 (5612): No heartbeat from core client for 30 sec - exiting 18:28:11 (5612): No heartbeat from core client for 30 sec - exiting 18:28:12 (5612): No heartbeat from core client for 30 sec - exiting 18:28:13 (5612): No heartbeat from core client for 30 sec - exiting 18:28:14 (5612): No heartbeat from core client for 30 sec - exiting 18:28:15 (5612): No heartbeat from core client for 30 sec - exiting 18:28:16 (5612): No heartbeat from core client for 30 sec - exiting 18:28:17 (5612): No heartbeat from core client for 30 sec - exiting 18:28:18 (5612): No heartbeat from core client for 30 sec - exiting 18:28:19 (5612): No heartbeat from core client for 30 sec - exiting 18:28:20 (5612): No heartbeat from core client for 30 sec - exiting 18:28:21 (5612): No heartbeat from core client for 30 sec - exiting 18:28:22 (5612): No heartbeat from core client for 30 sec - exiting 18:28:23 (5612): No heartbeat from core client for 30 sec - exiting 18:28:24 (5612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5560, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5516, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 23:27:44 (4628): No heartbeat from core client for 30 sec - exiting 23:27:46 (4628): No heartbeat from core client for 30 sec - exiting 23:27:47 (4628): No heartbeat from core client for 30 sec - exiting 23:27:48 (4628): No heartbeat from core client for 30 sec - exiting 23:27:49 (4628): No heartbeat from core client for 30 sec - exiting 23:27:50 (4628): No heartbeat from core client for 30 sec - exiting 23:27:51 (4628): No heartbeat from core client for 30 sec - exiting 23:27:52 (4628): No heartbeat from core client for 30 sec - exiting 23:27:53 (4628): No heartbeat from core client for 30 sec - exiting 23:27:54 (4628): No heartbeat from core client for 30 sec - exiting 23:27:55 (4628): No heartbeat from core client for 30 sec - exiting 23:27:56 (4628): No heartbeat from core client for 30 sec - exiting 23:27:57 (4628): No heartbeat from core client for 30 sec - exiting 23:27:58 (4628): No heartbeat from core client for 30 sec - exiting 23:27:59 (4628): No heartbeat from core client for 30 sec - exiting 23:28:00 (4628): No heartbeat from core client for 30 sec - exiting 23:28:01 (4628): No heartbeat from core client for 30 sec - exiting 23:28:02 (4628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4820, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5296, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6552, iMonCtr=1 Model crash detected, will try to restart... 21:13:50 (5260): No heartbeat from core client for 30 sec - exiting 21:13:51 (5260): No heartbeat from core client for 30 sec - exiting 21:13:52 (5260): No heartbeat from core client for 30 sec - exiting 21:13:53 (5260): No heartbeat from core client for 30 sec - exiting 21:13:54 (5260): No heartbeat from core client for 30 sec - exiting 21:13:55 (5260): No heartbeat from core client for 30 sec - exiting 21:13:56 (5260): No heartbeat from core client for 30 sec - exiting 21:13:57 (5260): No heartbeat from core client for 30 sec - exiting 21:13:58 (5260): No heartbeat from core client for 30 sec - exiting 21:13:59 (5260): No heartbeat from core client for 30 sec - exiting 21:14:00 (5260): No heartbeat from core client for 30 sec - exiting 21:14:01 (5260): No heartbeat from core client for 30 sec - exiting 21:14:02 (5260): No heartbeat from core client for 30 sec - exiting 21:14:03 (5260): No heartbeat from core client for 30 sec - exiting 21:14:04 (5260): No heartbeat from core client for 30 sec - exiting 21:14:05 (5260): No heartbeat from core client for 30 sec - exiting 21:14:06 (5260): No heartbeat from core client for 30 sec - exiting 21:14:07 (5260): No heartbeat from core client for 30 sec - exiting 21:14:08 (5260): No heartbeat from core client for 30 sec - exiting 21:14:09 (5260): No heartbeat from core client for 30 sec - exiting 21:14:10 (5260): No heartbeat from core client for 30 sec - exiting 21:14:11 (5260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:43:39 (3900): No heartbeat from core client for 30 sec - exiting 19:43:40 (3900): No heartbeat from core client for 30 sec - exiting 19:43:41 (3900): No heartbeat from core client for 30 sec - exiting 19:43:42 (3900): No heartbeat from core client for 30 sec - exiting 19:43:43 (3900): No heartbeat from core client for 30 sec - exiting 19:43:44 (3900): No heartbeat from core client for 30 sec - exiting 19:43:45 (3900): No heartbeat from core client for 30 sec - exiting 19:43:46 (3900): No heartbeat from core client for 30 sec - exiting 19:43:47 (3900): No heartbeat from core client for 30 sec - exiting 19:43:48 (3900): No heartbeat from core client for 30 sec - exiting 19:43:49 (3900): No heartbeat from core client for 30 sec - exiting 19:43:50 (3900): No heartbeat from core client for 30 sec - exiting 19:43:51 (3900): No heartbeat from core client for 30 sec - exiting 19:43:52 (3900): No heartbeat from core client for 30 sec - exiting 19:43:53 (3900): No heartbeat from core client for 30 sec - exiting 19:43:54 (3900): No heartbeat from core client for 30 sec - exiting 19:43:55 (3900): No heartbeat from core client for 30 sec - exiting 19:43:56 (3900): No heartbeat from core client for 30 sec - exiting 19:43:57 (3900): No heartbeat from core client for 30 sec - exiting 19:43:58 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:17:48 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5948, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3952, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5536, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5680, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 20:59:01 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:59:02 (5716): No heartbeat from core client for 30 sec - exiting 20:59:03 (5716): No heartbeat from core client for 30 sec - exiting 20:59:04 (5716): No heartbeat from core client for 30 sec - exiting 21:02:00 (7052): No heartbeat from core client for 30 sec - exiting 21:02:01 (7052): No heartbeat from core client for 30 sec - exiting 21:02:02 (7052): No heartbeat from core client for 30 sec - exiting 21:02:05 (7052): No heartbeat from core client for 30 sec - exiting 21:02:06 (7052): No heartbeat from core client for 30 sec - exiting 21:02:07 (7052): No heartbeat from core client for 30 sec - exiting 21:02:09 (7052): No heartbeat from core client for 30 sec - exiting 21:02:10 (7052): No heartbeat from core client for 30 sec - exiting 21:02:11 (7052): No heartbeat from core client for 30 sec - exiting 21:02:12 (7052): No heartbeat from core client for 30 sec - exiting 21:02:13 (7052): No heartbeat from core client for 30 sec - exiting 21:02:14 (7052): No heartbeat from core client for 30 sec - exiting 21:02:15 (7052): No heartbeat from core client for 30 sec - exiting 21:02:16 (7052): No heartbeat from core client for 30 sec - exiting 21:02:17 (7052): No heartbeat from core client for 30 sec - exiting 21:02:18 (7052): No heartbeat from core client for 30 sec - exiting 21:02:19 (7052): No heartbeat from core client for 30 sec - exiting 21:02:21 (7052): No heartbeat from core client for 30 sec - exiting 21:02:22 (7052): No heartbeat from core client for 30 sec - exiting 21:02:23 (7052): No heartbeat from core client for 30 sec - exiting 21:02:24 (7052): No heartbeat from core client for 30 sec - exiting 21:02:25 (7052): No heartbeat from core client for 30 sec - exiting 21:02:26 (7052): No heartbeat from core client for 30 sec - exiting 21:02:27 (7052): No heartbeat from core client for 30 sec - exiting 21:02:28 (7052): No heartbeat from core client for 30 sec - exiting 21:02:29 (7052): No heartbeat from core client for 30 sec - exiting 21:02:30 (7052): No heartbeat from core client for 30 sec - exiting 21:02:31 (7052): No heartbeat from core client for 30 sec - exiting 21:02:33 (7052): No heartbeat from core client for 30 sec - exiting 21:02:34 (7052): No heartbeat from core client for 30 sec - exiting 21:02:35 (7052): No heartbeat from core client for 30 sec - exiting 21:02:36 (7052): No heartbeat from core client for 30 sec - exiting 21:02:37 (7052): No heartbeat from core client for 30 sec - exiting 21:02:38 (7052): No heartbeat from core client for 30 sec - exiting 21:02:39 (7052): No heartbeat from core client for 30 sec - exiting 21:02:40 (7052): No heartbeat from core client for 30 sec - exiting 21:03:25 (7052): No heartbeat from core client for 30 sec - exiting 21:03:26 (7052): No heartbeat from core client for 30 sec - exiting 21:03:27 (7052): No heartbeat from core client for 30 sec - exiting 21:03:28 (7052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:25 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:14 (2344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:36:32 (5932): No heartbeat from core client for 30 sec - exiting 21:36:34 (5932): No heartbeat from core client for 30 sec - exiting 21:36:35 (5932): No heartbeat from core client for 30 sec - exiting 21:37:14 (5932): No heartbeat from core client for 30 sec - exiting 21:37:15 (5932): No heartbeat from core client for 30 sec - exiting 21:37:16 (5932): No heartbeat from core client for 30 sec - exiting 21:37:17 (5932): No heartbeat from core client for 30 sec - exiting 21:37:18 (5932): No heartbeat from core client for 30 sec - exiting 21:37:19 (5932): No heartbeat from core client for 30 sec - exiting 21:37:21 (5932): No heartbeat from core client for 30 sec - exiting 21:37:22 (5932): No heartbeat from core client for 30 sec - exiting 21:37:23 (5932): No heartbeat from core client for 30 sec - exiting 21:37:24 (5932): No heartbeat from core client for 30 sec - exiting 21:37:25 (5932): No heartbeat from core client for 30 sec - exiting 21:37:26 (5932): No heartbeat from core client for 30 sec - exiting 21:37:27 (5932): No heartbeat from core client for 30 sec - exiting 21:37:28 (5932): No heartbeat from core client for 30 sec - exiting 21:37:29 (5932): No heartbeat from core client for 30 sec - exiting 21:37:30 (5932): No heartbeat from core client for 30 sec - exiting 21:37:31 (5932): No heartbeat from core client for 30 sec - exiting 21:37:33 (5932): No heartbeat from core client for 30 sec - exiting 21:37:34 (5932): No heartbeat from core client for 30 sec - exiting 21:37:35 (5932): No heartbeat from core client for 30 sec - exiting 21:37:36 (5932): No heartbeat from core client for 30 sec - exiting 21:37:37 (5932): No heartbeat from core client for 30 sec - exiting 21:37:38 (5932): No heartbeat from core client for 30 sec - exiting 21:37:39 (5932): No heartbeat from core client for 30 sec - exiting 21:37:40 (5932): No heartbeat from core client for 30 sec - exiting 21:37:41 (5932): No heartbeat from core client for 30 sec - exiting 21:37:42 (5932): No heartbeat from core client for 30 sec - exiting 21:37:43 (5932): No heartbeat from core client for 30 sec - exiting 21:37:45 (5932): No heartbeat from core client for 30 sec - exiting 21:37:46 (5932): No heartbeat from core client for 30 sec - exiting 21:37:47 (5932): No heartbeat from core client for 30 sec - exiting 21:37:48 (5932): No heartbeat from core client for 30 sec - exiting 21:37:49 (5932): No heartbeat from core client for 30 sec - exiting 21:37:50 (5932): No heartbeat from core client for 30 sec - exiting 21:37:51 (5932): No heartbeat from core client for 30 sec - exiting 21:37:52 (5932): No heartbeat from core client for 30 sec - exiting 21:37:53 (5932): No heartbeat from core client for 30 sec - exiting 21:37:54 (5932): No heartbeat from core client for 30 sec - exiting 21:38:04 (5932): No heartbeat from core client for 30 sec - exiting 21:38:05 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:41:26 (6864): No heartbeat from core client for 30 sec - exiting 21:41:27 (6864): No heartbeat from core client for 30 sec - exiting 21:41:28 (6864): No heartbeat from core client for 30 sec - exiting 21:41:29 (6864): No heartbeat from core client for 30 sec - exiting 21:41:30 (6864): No heartbeat from core client for 30 sec - exiting 21:41:31 (6864): No heartbeat from core client for 30 sec - exiting 21:41:32 (6864): No heartbeat from core client for 30 sec - exiting 21:41:33 (6864): No heartbeat from core client for 30 sec - exiting 21:41:34 (6864): No heartbeat from core client for 30 sec - exiting 21:41:36 (6864): No heartbeat from core client for 30 sec - exiting 21:41:37 (6864): No heartbeat from core client for 30 sec - exiting 21:41:38 (6864): No heartbeat from core client for 30 sec - exiting 21:41:39 (6864): No heartbeat from core client for 30 sec - exiting 21:41:41 (6864): No heartbeat from core client for 30 sec - exiting 21:41:42 (6864): No heartbeat from core client for 30 sec - exiting 21:41:43 (6864): No heartbeat from core client for 30 sec - exiting 21:41:44 (6864): No heartbeat from core client for 30 sec - exiting 21:41:45 (6864): No heartbeat from core client for 30 sec - exiting 21:41:46 (6864): No heartbeat from core client for 30 sec - exiting 21:41:47 (6864): No heartbeat from core client for 30 sec - exiting 21:41:53 (6864): No heartbeat from core client for 30 sec - exiting 21:41:54 (6864): No heartbeat from core client for 30 sec - exiting 21:41:55 (6864): No heartbeat from core client for 30 sec - exiting 21:41:56 (6864): No heartbeat from core client for 30 sec - exiting 21:41:58 (6864): No heartbeat from core client for 30 sec - exiting 21:41:59 (6864): No heartbeat from core client for 30 sec - exiting 21:42:00 (6864): No heartbeat from core client for 30 sec - exiting 21:42:01 (6864): No heartbeat from core client for 30 sec - exiting 21:42:02 (6864): No heartbeat from core client for 30 sec - exiting 21:42:03 (6864): No heartbeat from core client for 30 sec - exiting 21:42:04 (6864): No heartbeat from core client for 30 sec - exiting 21:42:05 (6864): No heartbeat from core client for 30 sec - exiting 21:42:06 (6864): No heartbeat from core client for 30 sec - exiting 21:42:07 (6864): No heartbeat from core client for 30 sec - exiting 21:42:08 (6864): No heartbeat from core client for 30 sec - exiting 21:42:10 (6864): No heartbeat from core client for 30 sec - exiting 21:42:11 (6864): No heartbeat from core client for 30 sec - exiting 21:42:12 (6864): No heartbeat from core client for 30 sec - exiting 21:42:13 (6864): No heartbeat from core client for 30 sec - exiting 21:42:14 (6864): No heartbeat from core client for 30 sec - exiting 21:42:15 (6864): No heartbeat from core client for 30 sec - exiting 21:42:16 (6864): No heartbeat from core client for 30 sec - exiting 21:42:17 (6864): No heartbeat from core client for 30 sec - exiting 21:42:39 (6864): No heartbeat from core client for 30 sec - exiting 21:42:41 (6864): No heartbeat from core client for 30 sec - exiting 21:42:42 (6864): No heartbeat from core client for 30 sec - exiting 21:42:44 (6864): No heartbeat from core client for 30 sec - exiting 21:42:45 (6864): No heartbeat from core client for 30 sec - exiting 21:42:46 (6864): No heartbeat from core client for 30 sec - exiting 21:42:47 (6864): No heartbeat from core client for 30 sec - exiting 21:42:48 (6864): No heartbeat from core client for 30 sec - exiting 21:42:49 (6864): No heartbeat from core client for 30 sec - exiting 21:42:50 (6864): No heartbeat from core client for 30 sec - exiting 21:42:51 (6864): No heartbeat from core client for 30 sec - exiting 21:42:52 (6864): No heartbeat from core client for 30 sec - exiting 21:42:53 (6864): No heartbeat from core client for 30 sec - exiting 21:42:54 (6864): No heartbeat from core client for 30 sec - exiting 21:42:56 (6864): No heartbeat from core client for 30 sec - exiting 21:42:57 (6864): No heartbeat from core client for 30 sec - exiting 21:42:58 (6864): No heartbeat from core client for 30 sec - exiting 21:42:59 (6864): No heartbeat from core client for 30 sec - exiting 21:43:00 (6864): No heartbeat from core client for 30 sec - exiting 21:43:01 (6864): No heartbeat from core client for 30 sec - exiting 21:43:02 (6864): No heartbeat from core client for 30 sec - exiting 21:43:03 (6864): No heartbeat from core client for 30 sec - exiting 21:43:04 (6864): No heartbeat from core client for 30 sec - exiting 21:43:05 (6864): No heartbeat from core client for 30 sec - exiting 21:43:06 (6864): No heartbeat from core client for 30 sec - exiting 21:43:08 (6864): No heartbeat from core client for 30 sec - exiting 21:43:09 (6864): No heartbeat from core client for 30 sec - exiting 21:43:10 (6864): No heartbeat from core client for 30 sec - exiting 21:43:11 (6864): No heartbeat from core client for 30 sec - exiting 21:43:12 (6864): No heartbeat from core client for 30 sec - exiting 21:43:13 (6864): No heartbeat from core client for 30 sec - exiting 21:43:14 (6864): No heartbeat from core client for 30 sec - exiting 21:43:15 (6864): No heartbeat from core client for 30 sec - exiting 21:43:16 (6864): No heartbeat from core client for 30 sec - exiting 21:43:17 (6864): No heartbeat from core client for 30 sec - exiting 21:43:18 (6864): No heartbeat from core client for 30 sec - exiting 21:43:20 (6864): No heartbeat from core client for 30 sec - exiting 21:43:21 (6864): No heartbeat from core client for 30 sec - exiting 21:43:22 (6864): No heartbeat from core client for 30 sec - exiting 21:43:23 (6864): No heartbeat from core client for 30 sec - exiting 21:43:24 (6864): No heartbeat from core client for 30 sec - exiting 21:43:25 (6864): No heartbeat from core client for 30 sec - exiting 21:43:26 (6864): No heartbeat from core client for 30 sec - exiting 21:43:27 (6864): No heartbeat from core client for 30 sec - exiting 21:43:28 (6864): No heartbeat from core client for 30 sec - exiting 21:43:29 (6864): No heartbeat from core client for 30 sec - exiting 21:43:30 (6864): No heartbeat from core client for 30 sec - exiting 21:43:32 (6864): No heartbeat from core client for 30 sec - exiting 21:43:33 (6864): No heartbeat from core client for 30 sec - exiting 21:43:34 (6864): No heartbeat from core client for 30 sec - exiting 21:43:35 (6864): No heartbeat from core client for 30 sec - exiting 21:43:36 (6864): No heartbeat from core client for 30 sec - exiting 21:43:37 (6864): No heartbeat from core client for 30 sec - exiting 21:43:38 (6864): No heartbeat from core client for 30 sec - exiting 21:43:41 (6864): No heartbeat from core client for 30 sec - exiting 21:43:42 (6864): No heartbeat from core client for 30 sec - exiting 21:43:43 (6864): No heartbeat from core client for 30 sec - exiting 21:43:44 (6864): No heartbeat from core client for 30 sec - exiting 21:43:45 (6864): No heartbeat from core client for 30 sec - exiting 21:43:46 (6864): No heartbeat from core client for 30 sec - exiting 21:43:48 (6864): No heartbeat from core client for 30 sec - exiting 21:43:49 (6864): No heartbeat from core client for 30 sec - exiting 21:43:50 (6864): No heartbeat from core client for 30 sec - exiting 21:43:51 (6864): No heartbeat from core client for 30 sec - exiting 21:43:52 (6864): No heartbeat from core client for 30 sec - exiting 21:43:53 (6864): No heartbeat from core client for 30 sec - exiting 21:43:54 (6864): No heartbeat from core client for 30 sec - exiting 21:43:56 (6864): No heartbeat from core client for 30 sec - exiting 21:43:57 (6864): No heartbeat from core client for 30 sec - exiting 21:43:58 (6864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5416, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:47:42 (5040): No heartbeat from core client for 30 sec - exiting 22:47:43 (5040): No heartbeat from core client for 30 sec - exiting 22:47:44 (5040): No heartbeat from core client for 30 sec - exiting 22:47:45 (5040): No heartbeat from core client for 30 sec - exiting 22:47:46 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:47:47 (5040): No heartbeat from core client for 30 sec - exiting 23:26:52 (8152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:27:34 (6824): No heartbeat from core client for 30 sec - exiting 23:27:35 (6824): No heartbeat from core client for 30 sec - exiting 23:27:36 (6824): No heartbeat from core client for 30 sec - exiting 23:27:37 (6824): No heartbeat from core client for 30 sec - exiting 23:27:38 (6824): No heartbeat from core client for 30 sec - exiting 23:27:39 (6824): No heartbeat from core client for 30 sec - exiting 23:27:40 (6824): No heartbeat from core client for 30 sec - exiting 23:27:41 (6824): No heartbeat from core client for 30 sec - exiting 23:27:42 (6824): No heartbeat from core client for 30 sec - exiting 23:27:43 (6824): No heartbeat from core client for 30 sec - exiting 23:27:44 (6824): No heartbeat from core client for 30 sec - exiting 23:27:45 (6824): No heartbeat from core client for 30 sec - exiting 23:27:46 (6824): No heartbeat from core client for 30 sec - exiting 23:27:47 (6824): No heartbeat from core client for 30 sec - exiting 23:27:48 (6824): No heartbeat from core client for 30 sec - exiting 23:27:49 (6824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6540, iMonCtr=1 Model crash detected, will try to restart... 15:58:41 (6076): No heartbeat from core client for 30 sec - exiting 15:58:42 (6076): No heartbeat from core client for 30 sec - exiting 15:58:44 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:29 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7752FF6B write attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 22:46:37 (6104): No heartbeat from core client for 30 sec - exiting 22:46:39 (6104): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77607203 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Dec 2013 11:35:12	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	1,036,800	1,625,778	1.5681
08 Dec 2013 14:01:01	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	1,010,880	1,586,125	1.5691
07 Dec 2013 16:30:46	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	984,960	1,546,438	1.5701
06 Dec 2013 21:41:33	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	959,040	1,505,745	1.5701
06 Dec 2013 12:09:13	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	933,120	1,467,081	1.5722
06 Dec 2013 12:09:13	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	907,200	1,432,093	1.5786
06 Dec 2013 12:09:13	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	881,280	1,395,258	1.5832
21 Nov 2013 12:44:58	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	855,360	1,357,095	1.5866
21 Nov 2013 12:44:58	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	829,440	1,318,666	1.5898
21 Nov 2013 12:44:58	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	803,520	1,278,801	1.5915
11 Nov 2013 12:57:53	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	777,600	1,239,168	1.5936
08 Nov 2013 21:33:32	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	751,680	1,198,345	1.5942
07 Nov 2013 17:06:54	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	725,760	1,157,438	1.5948
06 Nov 2013 15:55:10	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	699,840	1,116,437	1.5953
05 Nov 2013 16:00:40	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	673,920	1,075,675	1.5961
04 Nov 2013 15:32:38	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	648,000	1,035,454	1.5979
03 Nov 2013 18:13:37	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	622,080	994,845	1.5992
03 Nov 2013 06:27:34	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	596,160	954,462	1.6010
02 Nov 2013 15:29:05	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	570,240	913,814	1.6025
01 Nov 2013 14:09:35	1202025	16052215	hadcm3n_ocyt_1900_40_008471928_2	544,320	873,319	1.6044