Task 15277333

Name	hadcm3n_zmr8_1880_40_008199682_0
Workunit	8354806
Created	13 Sep 2012, 2:17:57 UTC
Sent	14 Sep 2012, 14:30:46 UTC
Report deadline	14 Dec 2012, 21:57:57 UTC
Received	1 Nov 2012, 18:21:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1193903
Run time	5 days 12 hours 23 min 58 sec
CPU time	5 days 8 hours 20 min 33 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5112, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5032, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4516, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4740, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2504, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5612, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4912, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4912, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4912, iMonCtr=1 Model crash detected, will try to restart... 20:38:54 (4744): No heartbeat from core client for 30 sec - exiting 20:38:55 (4744): No heartbeat from core client for 30 sec - exiting 20:38:56 (4744): No heartbeat from core client for 30 sec - exiting 20:38:57 (4744): No heartbeat from core client for 30 sec - exiting 20:38:58 (4744): No heartbeat from core client for 30 sec - exiting 20:38:59 (4744): No heartbeat from core client for 30 sec - exiting 20:39:00 (4744): No heartbeat from core client for 30 sec - exiting 20:39:01 (4744): No heartbeat from core client for 30 sec - exiting 20:39:02 (4744): No heartbeat from core client for 30 sec - exiting 20:39:04 (4744): No heartbeat from core client for 30 sec - exiting 20:39:05 (4744): No heartbeat from core client for 30 sec - exiting 20:39:06 (4744): No heartbeat from core client for 30 sec - exiting 20:39:07 (4744): No heartbeat from core client for 30 sec - exiting 20:39:08 (4744): No heartbeat from core client for 30 sec - exiting 20:39:09 (4744): No heartbeat from core client for 30 sec - exiting 20:39:10 (4744): No heartbeat from core client for 30 sec - exiting 20:39:11 (4744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:07 (760): No heartbeat from core client for 30 sec - exiting 09:08:09 (760): No heartbeat from core client for 30 sec - exiting 09:08:10 (760): No heartbeat from core client for 30 sec - exiting 09:08:11 (760): No heartbeat from core client for 30 sec - exiting 09:08:12 (760): No heartbeat from core client for 30 sec - exiting 09:08:13 (760): No heartbeat from core client for 30 sec - exiting 09:08:14 (760): No heartbeat from core client for 30 sec - exiting 09:08:15 (760): No heartbeat from core client for 30 sec - exiting 09:08:16 (760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1956, iMonCtr=1 Model crash detected, will try to restart... 10:26:46 (4072): No heartbeat from core client for 30 sec - exiting 10:26:47 (4072): No heartbeat from core client for 30 sec - exiting 10:26:48 (4072): No heartbeat from core client for 30 sec - exiting 10:26:49 (4072): No heartbeat from core client for 30 sec - exiting 10:26:50 (4072): No heartbeat from core client for 30 sec - exiting 10:26:51 (4072): No heartbeat from core client for 30 sec - exiting 10:26:52 (4072): No heartbeat from core client for 30 sec - exiting 10:26:53 (4072): No heartbeat from core client for 30 sec - exiting 10:26:54 (4072): No heartbeat from core client for 30 sec - exiting 10:26:55 (4072): No heartbeat from core client for 30 sec - exiting 10:26:57 (4072): No heartbeat from core client for 30 sec - exiting 10:26:58 (4072): No heartbeat from core client for 30 sec - exiting 10:26:59 (4072): No heartbeat from core client for 30 sec - exiting 10:27:00 (4072): No heartbeat from core client for 30 sec - exiting 10:27:01 (4072): No heartbeat from core client for 30 sec - exiting 10:27:02 (4072): No heartbeat from core client for 30 sec - exiting 10:27:03 (4072): No heartbeat from core client for 30 sec - exiting 10:27:04 (4072): No heartbeat from core client for 30 sec - exiting 10:27:05 (4072): No heartbeat from core client for 30 sec - exiting 10:27:06 (4072): No heartbeat from core client for 30 sec - exiting 10:27:07 (4072): No heartbeat from core client for 30 sec - exiting 10:27:09 (4072): No heartbeat from core client for 30 sec - exiting 10:27:10 (4072): No heartbeat from core client for 30 sec - exiting 10:27:11 (4072): No heartbeat from core client for 30 sec - exiting 10:27:12 (4072): No heartbeat from core client for 30 sec - exiting 10:27:13 (4072): No heartbeat from core client for 30 sec - exiting 10:27:14 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:36:18 (3468): No heartbeat from core client for 30 sec - exiting 21:36:20 (3468): No heartbeat from core client for 30 sec - exiting 21:36:21 (3468): No heartbeat from core client for 30 sec - exiting 21:36:22 (3468): No heartbeat from core client for 30 sec - exiting 21:36:23 (3468): No heartbeat from core client for 30 sec - exiting 21:36:24 (3468): No heartbeat from core client for 30 sec - exiting 21:36:25 (3468): No heartbeat from core client for 30 sec - exiting 21:36:26 (3468): No heartbeat from core client for 30 sec - exiting 21:36:28 (3468): No heartbeat from core client for 30 sec - exiting 21:36:29 (3468): No heartbeat from core client for 30 sec - exiting 21:36:30 (3468): No heartbeat from core client for 30 sec - exiting 21:36:31 (3468): No heartbeat from core client for 30 sec - exiting 21:36:32 (3468): No heartbeat from core client for 30 sec - exiting 21:36:33 (3468): No heartbeat from core client for 30 sec - exiting 21:36:34 (3468): No heartbeat from core client for 30 sec - exiting 21:36:35 (3468): No heartbeat from core client for 30 sec - exiting 21:36:36 (3468): No heartbeat from core client for 30 sec - exiting 21:36:37 (3468): No heartbeat from core client for 30 sec - exiting 21:36:38 (3468): No heartbeat from core client for 30 sec - exiting 21:36:40 (3468): No heartbeat from core client for 30 sec - exiting 21:36:41 (3468): No heartbeat from core client for 30 sec - exiting 21:36:42 (3468): No heartbeat from core client for 30 sec - exiting 21:36:43 (3468): No heartbeat from core client for 30 sec - exiting 21:36:44 (3468): No heartbeat from core client for 30 sec - exiting 21:36:45 (3468): No heartbeat from core client for 30 sec - exiting 21:36:46 (3468): No heartbeat from core client for 30 sec - exiting 21:36:47 (3468): No heartbeat from core client for 30 sec - exiting 21:36:48 (3468): No heartbeat from core client for 30 sec - exiting 21:36:49 (3468): No heartbeat from core client for 30 sec - exiting 21:36:50 (3468): No heartbeat from core client for 30 sec - exiting 21:36:51 (3468): No heartbeat from core client for 30 sec - exiting 21:36:53 (3468): No heartbeat from core client for 30 sec - exiting 21:36:54 (3468): No heartbeat from core client for 30 sec - exiting 21:36:55 (3468): No heartbeat from core client for 30 sec - exiting 21:36:56 (3468): No heartbeat from core client for 30 sec - exiting 21:36:57 (3468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:44:50 (2552): No heartbeat from core client for 30 sec - exiting 06:44:51 (2552): No heartbeat from core client for 30 sec - exiting 06:44:52 (2552): No heartbeat from core client for 30 sec - exiting 06:44:53 (2552): No heartbeat from core client for 30 sec - exiting 06:44:55 (2552): No heartbeat from core client for 30 sec - exiting 06:44:56 (2552): No heartbeat from core client for 30 sec - exiting 06:44:57 (2552): No heartbeat from core client for 30 sec - exiting 06:44:58 (2552): No heartbeat from core client for 30 sec - exiting 06:44:59 (2552): No heartbeat from core client for 30 sec - exiting 06:45:00 (2552): No heartbeat from core client for 30 sec - exiting 06:45:01 (2552): No heartbeat from core client for 30 sec - exiting 06:45:02 (2552): No heartbeat from core client for 30 sec - exiting 06:45:03 (2552): No heartbeat from core client for 30 sec - exiting 06:45:04 (2552): No heartbeat from core client for 30 sec - exiting 06:45:06 (2552): No heartbeat from core client for 30 sec - exiting 06:45:07 (2552): No heartbeat from core client for 30 sec - exiting 06:45:08 (2552): No heartbeat from core client for 30 sec - exiting 06:45:09 (2552): No heartbeat from core client for 30 sec - exiting 06:45:10 (2552): No heartbeat from core client for 30 sec - exiting 06:45:11 (2552): No heartbeat from core client for 30 sec - exiting 06:45:12 (2552): No heartbeat from core client for 30 sec - exiting 06:45:13 (2552): No heartbeat from core client for 30 sec - exiting 06:45:14 (2552): No heartbeat from core client for 30 sec - exiting 06:45:16 (2552): No heartbeat from core client for 30 sec - exiting 06:45:17 (2552): No heartbeat from core client for 30 sec - exiting 06:45:18 (2552): No heartbeat from core client for 30 sec - exiting 06:45:19 (2552): No heartbeat from core client for 30 sec - exiting 06:45:20 (2552): No heartbeat from core client for 30 sec - exiting 06:45:21 (2552): No heartbeat from core client for 30 sec - exiting 06:45:22 (2552): No heartbeat from core client for 30 sec - exiting 06:45:23 (2552): No heartbeat from core client for 30 sec - exiting 06:45:24 (2552): No heartbeat from core client for 30 sec - exiting 06:45:25 (2552): No heartbeat from core client for 30 sec - exiting 06:45:27 (2552): No heartbeat from core client for 30 sec - exiting 06:45:28 (2552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:10 (5144): No heartbeat from core client for 30 sec - exiting 12:46:11 (5144): No heartbeat from core client for 30 sec - exiting 12:46:12 (5144): No heartbeat from core client for 30 sec - exiting 12:46:13 (5144): No heartbeat from core client for 30 sec - exiting 12:46:14 (5144): No heartbeat from core client for 30 sec - exiting 12:46:15 (5144): No heartbeat from core client for 30 sec - exiting 12:46:16 (5144): No heartbeat from core client for 30 sec - exiting 12:46:17 (5144): No heartbeat from core client for 30 sec - exiting 12:46:18 (5144): No heartbeat from core client for 30 sec - exiting 12:46:20 (5144): No heartbeat from core client for 30 sec - exiting 12:46:21 (5144): No heartbeat from core client for 30 sec - exiting 12:46:22 (5144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:23 (5144): No heartbeat from core client for 30 sec - exiting 16:39:38 (5344): No heartbeat from core client for 30 sec - exiting 16:39:39 (5344): No heartbeat from core client for 30 sec - exiting 16:39:41 (5344): No heartbeat from core client for 30 sec - exiting 16:39:42 (5344): No heartbeat from core client for 30 sec - exiting 16:39:43 (5344): No heartbeat from core client for 30 sec - exiting 16:39:44 (5344): No heartbeat from core client for 30 sec - exiting 16:39:45 (5344): No heartbeat from core client for 30 sec - exiting 16:39:46 (5344): No heartbeat from core client for 30 sec - exiting 16:39:47 (5344): No heartbeat from core client for 30 sec - exiting 16:39:48 (5344): No heartbeat from core client for 30 sec - exiting 16:39:49 (5344): No heartbeat from core client for 30 sec - exiting 16:39:50 (5344): No heartbeat from core client for 30 sec - exiting 16:39:51 (5344): No heartbeat from core client for 30 sec - exiting 16:39:53 (5344): No heartbeat from core client for 30 sec - exiting 16:39:54 (5344): No heartbeat from core client for 30 sec - exiting 16:39:55 (5344): No heartbeat from core client for 30 sec - exiting 16:39:56 (5344): No heartbeat from core client for 30 sec - exiting 16:39:57 (5344): No heartbeat from core client for 30 sec - exiting 16:39:58 (5344): No heartbeat from core client for 30 sec - exiting 16:39:59 (5344): No heartbeat from core client for 30 sec - exiting 16:40:00 (5344): No heartbeat from core client for 30 sec - exiting 16:40:01 (5344): No heartbeat from core client for 30 sec - exiting 16:40:03 (5344): No heartbeat from core client for 30 sec - exiting 16:40:04 (5344): No heartbeat from core client for 30 sec - exiting 16:40:05 (5344): No heartbeat from core client for 30 sec - exiting 16:40:06 (5344): No heartbeat from core client for 30 sec - exiting 16:40:07 (5344): No heartbeat from core client for 30 sec - exiting 16:40:08 (5344): No heartbeat from core client for 30 sec - exiting 16:40:10 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:55:50 (5240): No heartbeat from core client for 30 sec - exiting 06:55:55 (5240): No heartbeat from core client for 30 sec - exiting 06:55:56 (5240): No heartbeat from core client for 30 sec - exiting 06:55:57 (5240): No heartbeat from core client for 30 sec - exiting 06:55:58 (5240): No heartbeat from core client for 30 sec - exiting 06:55:59 (5240): No heartbeat from core client for 30 sec - exiting 06:56:00 (5240): No heartbeat from core client for 30 sec - exiting 06:56:01 (5240): No heartbeat from core client for 30 sec - exiting 06:56:02 (5240): No heartbeat from core client for 30 sec - exiting 06:56:03 (5240): No heartbeat from core client for 30 sec - exiting 06:56:04 (5240): No heartbeat from core client for 30 sec - exiting 06:56:06 (5240): No heartbeat from core client for 30 sec - exiting 06:56:07 (5240): No heartbeat from core client for 30 sec - exiting 06:56:08 (5240): No heartbeat from core client for 30 sec - exiting 06:56:09 (5240): No heartbeat from core client for 30 sec - exiting 06:56:10 (5240): No heartbeat from core client for 30 sec - exiting 06:56:11 (5240): No heartbeat from core client for 30 sec - exiting 06:56:12 (5240): No heartbeat from core client for 30 sec - exiting 06:56:13 (5240): No heartbeat from core client for 30 sec - exiting 06:56:14 (5240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:57:54 (3536): No heartbeat from core client for 30 sec - exiting 15:57:55 (3536): No heartbeat from core client for 30 sec - exiting 15:57:56 (3536): No heartbeat from core client for 30 sec - exiting 15:57:57 (3536): No heartbeat from core client for 30 sec - exiting 15:57:58 (3536): No heartbeat from core client for 30 sec - exiting 15:57:59 (3536): No heartbeat from core client for 30 sec - exiting 15:58:00 (3536): No heartbeat from core client for 30 sec - exiting 15:58:02 (3536): No heartbeat from core client for 30 sec - exiting 15:58:03 (3536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:07:48 (4808): No heartbeat from core client for 30 sec - exiting 20:07:49 (4808): No heartbeat from core client for 30 sec - exiting 20:07:50 (4808): No heartbeat from core client for 30 sec - exiting 20:07:51 (4808): No heartbeat from core client for 30 sec - exiting 20:07:52 (4808): No heartbeat from core client for 30 sec - exiting 20:07:53 (4808): No heartbeat from core client for 30 sec - exiting 20:07:54 (4808): No heartbeat from core client for 30 sec - exiting 20:07:55 (4808): No heartbeat from core client for 30 sec - exiting 20:07:56 (4808): No heartbeat from core client for 30 sec - exiting 20:07:57 (4808): No heartbeat from core client for 30 sec - exiting 20:07:59 (4808): No heartbeat from core client for 30 sec - exiting 20:08:00 (4808): No heartbeat from core client for 30 sec - exiting 20:08:01 (4808): No heartbeat from core client for 30 sec - exiting 20:08:02 (4808): No heartbeat from core client for 30 sec - exiting 20:08:03 (4808): No heartbeat from core client for 30 sec - exiting 20:08:04 (4808): No heartbeat from core client for 30 sec - exiting 20:08:05 (4808): No heartbeat from core client for 30 sec - exiting 20:08:06 (4808): No heartbeat from core client for 30 sec - exiting 20:08:07 (4808): No heartbeat from core client for 30 sec - exiting 20:08:08 (4808): No heartbeat from core client for 30 sec - exiting 20:08:09 (4808): No heartbeat from core client for 30 sec - exiting 20:08:10 (4808): No heartbeat from core client for 30 sec - exiting 20:08:11 (4808): No heartbeat from core client for 30 sec - exiting 20:08:12 (4808): No heartbeat from core client for 30 sec - exiting 20:08:13 (4808): No heartbeat from core client for 30 sec - exiting 20:08:14 (4808): No heartbeat from core client for 30 sec - exiting 20:08:15 (4808): No heartbeat from core client for 30 sec - exiting 20:08:16 (4808): No heartbeat from core client for 30 sec - exiting 20:08:17 (4808): No heartbeat from core client for 30 sec - exiting 20:08:18 (4808): No heartbeat from core client for 30 sec - exiting 20:08:19 (4808): No heartbeat from core client for 30 sec - exiting 20:08:20 (4808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:21:30 (4792): No heartbeat from core client for 30 sec - exiting 12:21:31 (4792): No heartbeat from core client for 30 sec - exiting 12:21:32 (4792): No heartbeat from core client for 30 sec - exiting 12:21:33 (4792): No heartbeat from core client for 30 sec - exiting 12:21:35 (4792): No heartbeat from core client for 30 sec - exiting 12:21:36 (4792): No heartbeat from core client for 30 sec - exiting 12:21:37 (4792): No heartbeat from core client for 30 sec - exiting 12:21:38 (4792): No heartbeat from core client for 30 sec - exiting 12:21:39 (4792): No heartbeat from core client for 30 sec - exiting 12:21:40 (4792): No heartbeat from core client for 30 sec - exiting 12:21:41 (4792): No heartbeat from core client for 30 sec - exiting 12:21:42 (4792): No heartbeat from core client for 30 sec - exiting 12:21:43 (4792): No heartbeat from core client for 30 sec - exiting 12:21:44 (4792): No heartbeat from core client for 30 sec - exiting 12:21:46 (4792): No heartbeat from core client for 30 sec - exiting 12:21:47 (4792): No heartbeat from core client for 30 sec - exiting 12:21:48 (4792): No heartbeat from core client for 30 sec - exiting 12:21:49 (4792): No heartbeat from core client for 30 sec - exiting 12:21:50 (4792): No heartbeat from core client for 30 sec - exiting 12:21:51 (4792): No heartbeat from core client for 30 sec - exiting 12:21:52 (4792): No heartbeat from core client for 30 sec - exiting 12:21:53 (4792): No heartbeat from core client for 30 sec - exiting 12:21:54 (4792): No heartbeat from core client for 30 sec - exiting 12:21:55 (4792): No heartbeat from core client for 30 sec - exiting 12:21:57 (4792): No heartbeat from core client for 30 sec - exiting 12:21:58 (4792): No heartbeat from core client for 30 sec - exiting 12:21:59 (4792): No heartbeat from core client for 30 sec - exiting 12:22:00 (4792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:38:40 (4788): No heartbeat from core client for 30 sec - exiting 19:38:41 (4788): No heartbeat from core client for 30 sec - exiting 19:38:42 (4788): No heartbeat from core client for 30 sec - exiting 19:38:43 (4788): No heartbeat from core client for 30 sec - exiting 19:38:44 (4788): No heartbeat from core client for 30 sec - exiting 19:38:45 (4788): No heartbeat from core client for 30 sec - exiting 19:38:46 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3944, iMonCtr=1 Model crash detected, will try to restart... 19:00:00 (4752): No heartbeat from core client for 30 sec - exiting 19:00:01 (4752): No heartbeat from core client for 30 sec - exiting 19:00:02 (4752): No heartbeat from core client for 30 sec - exiting 19:00:03 (4752): No heartbeat from core client for 30 sec - exiting 19:00:04 (4752): No heartbeat from core client for 30 sec - exiting 19:00:05 (4752): No heartbeat from core client for 30 sec - exiting 19:00:07 (4752): No heartbeat from core client for 30 sec - exiting 19:00:08 (4752): No heartbeat from core client for 30 sec - exiting 19:00:09 (4752): No heartbeat from core client for 30 sec - exiting 19:00:10 (4752): No heartbeat from core client for 30 sec - exiting 19:00:11 (4752): No heartbeat from core client for 30 sec - exiting 19:00:12 (4752): No heartbeat from core client for 30 sec - exiting 19:00:13 (4752): No heartbeat from core client for 30 sec - exiting 19:00:14 (4752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:32:06 (5620): No heartbeat from core client for 30 sec - exiting 10:32:07 (5620): No heartbeat from core client for 30 sec - exiting 10:32:08 (5620): No heartbeat from core client for 30 sec - exiting 10:32:09 (5620): No heartbeat from core client for 30 sec - exiting 10:32:10 (5620): No heartbeat from core client for 30 sec - exiting 10:32:11 (5620): No heartbeat from core client for 30 sec - exiting 10:32:12 (5620): No heartbeat from core client for 30 sec - exiting 10:32:13 (5620): No heartbeat from core client for 30 sec - exiting 10:32:14 (5620): No heartbeat from core client for 30 sec - exiting 10:32:15 (5620): No heartbeat from core client for 30 sec - exiting 10:32:16 (5620): No heartbeat from core client for 30 sec - exiting 10:32:17 (5620): No heartbeat from core client for 30 sec - exiting 10:32:18 (5620): No heartbeat from core client for 30 sec - exiting 10:32:19 (5620): No heartbeat from core client for 30 sec - exiting 10:32:20 (5620): No heartbeat from core client for 30 sec - exiting 10:32:21 (5620): No heartbeat from core client for 30 sec - exiting 10:32:22 (5620): No heartbeat from core client for 30 sec - exiting 10:32:23 (5620): No heartbeat from core client for 30 sec - exiting 10:32:24 (5620): No heartbeat from core client for 30 sec - exiting 10:32:25 (5620): No heartbeat from core client for 30 sec - exiting 10:32:26 (5620): No heartbeat from core client for 30 sec - exiting 10:32:27 (5620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:17:56 (4520): No heartbeat from core client for 30 sec - exiting 21:17:57 (4520): No heartbeat from core client for 30 sec - exiting 21:17:58 (4520): No heartbeat from core client for 30 sec - exiting 21:18:00 (4520): No heartbeat from core client for 30 sec - exiting 21:18:01 (4520): No heartbeat from core client for 30 sec - exiting 21:18:02 (4520): No heartbeat from core client for 30 sec - exiting 21:18:03 (4520): No heartbeat from core client for 30 sec - exiting 21:18:04 (4520): No heartbeat from core client for 30 sec - exiting 21:18:05 (4520): No heartbeat from core client for 30 sec - exiting 21:18:06 (4520): No heartbeat from core client for 30 sec - exiting 21:18:07 (4520): No heartbeat from core client for 30 sec - exiting 21:18:08 (4520): No heartbeat from core client for 30 sec - exiting 21:18:10 (4520): No heartbeat from core client for 30 sec - exiting 21:18:11 (4520): No heartbeat from core client for 30 sec - exiting 21:18:12 (4520): No heartbeat from core client for 30 sec - exiting 21:18:13 (4520): No heartbeat from core client for 30 sec - exiting 21:18:14 (4520): No heartbeat from core client for 30 sec - exiting 21:18:15 (4520): No heartbeat from core client for 30 sec - exiting 21:18:16 (4520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:55:02 (2124): No heartbeat from core client for 30 sec - exiting 06:55:03 (2124): No heartbeat from core client for 30 sec - exiting 06:55:04 (2124): No heartbeat from core client for 30 sec - exiting 06:55:06 (2124): No heartbeat from core client for 30 sec - exiting 06:55:07 (2124): No heartbeat from core client for 30 sec - exiting 06:55:08 (2124): No heartbeat from core client for 30 sec - exiting 06:55:09 (2124): No heartbeat from core client for 30 sec - exiting 06:55:10 (2124): No heartbeat from core client for 30 sec - exiting 06:55:11 (2124): No heartbeat from core client for 30 sec - exiting 06:55:12 (2124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:04:17 (4420): No heartbeat from core client for 30 sec - exiting 08:04:18 (4420): No heartbeat from core client for 30 sec - exiting 08:04:19 (4420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=1 Model crash detected, will try to restart... 12:30:40 (4672): No heartbeat from core client for 30 sec - exiting 12:30:41 (4672): No heartbeat from core client for 30 sec - exiting 12:30:42 (4672): No heartbeat from core client for 30 sec - exiting 12:30:43 (4672): No heartbeat from core client for 30 sec - exiting 12:30:44 (4672): No heartbeat from core client for 30 sec - exiting 12:30:45 (4672): No heartbeat from core client for 30 sec - exiting 12:30:46 (4672): No heartbeat from core client for 30 sec - exiting 12:30:47 (4672): No heartbeat from core client for 30 sec - exiting 12:30:49 (4672): No heartbeat from core client for 30 sec - exiting 12:30:50 (4672): No heartbeat from core client for 30 sec - exiting 12:30:51 (4672): No heartbeat from core client for 30 sec - exiting 12:30:52 (4672): No heartbeat from core client for 30 sec - exiting 12:30:53 (4672): No heartbeat from core client for 30 sec - exiting 12:30:54 (4672): No heartbeat from core client for 30 sec - exiting 12:30:55 (4672): No heartbeat from core client for 30 sec - exiting 12:30:56 (4672): No heartbeat from core client for 30 sec - exiting 12:30:57 (4672): No heartbeat from core client for 30 sec - exiting 12:30:58 (4672): No heartbeat from core client for 30 sec - exiting 12:30:59 (4672): No heartbeat from core client for 30 sec - exiting 12:31:00 (4672): No heartbeat from core client for 30 sec - exiting 12:31:01 (4672): No heartbeat from core client for 30 sec - exiting 12:31:03 (4672): No heartbeat from core client for 30 sec - exiting 12:31:04 (4672): No heartbeat from core client for 30 sec - exiting 12:31:05 (4672): No heartbeat from core client for 30 sec - exiting 12:31:06 (4672): No heartbeat from core client for 30 sec - exiting 12:31:07 (4672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:15:35 (4336): No heartbeat from core client for 30 sec - exiting 17:15:36 (4336): No heartbeat from core client for 30 sec - exiting 17:15:37 (4336): No heartbeat from core client for 30 sec - exiting 17:15:38 (4336): No heartbeat from core client for 30 sec - exiting 17:15:40 (4336): No heartbeat from core client for 30 sec - exiting 17:15:41 (4336): No heartbeat from core client for 30 sec - exiting 17:15:42 (4336): No heartbeat from core client for 30 sec - exiting 17:15:43 (4336): No heartbeat from core client for 30 sec - exiting 17:15:44 (4336): No heartbeat from core client for 30 sec - exiting 17:15:45 (4336): No heartbeat from core client for 30 sec - exiting 17:15:46 (4336): No heartbeat from core client for 30 sec - exiting 17:15:47 (4336): No heartbeat from core client for 30 sec - exiting 17:15:48 (4336): No heartbeat from core client for 30 sec - exiting 17:15:49 (4336): No heartbeat from core client for 30 sec - exiting 17:15:50 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:50:42 (4684): No heartbeat from core client for 30 sec - exiting 10:50:44 (4684): No heartbeat from core client for 30 sec - exiting 10:50:45 (4684): No heartbeat from core client for 30 sec - exiting 10:50:46 (4684): No heartbeat from core client for 30 sec - exiting 10:50:47 (4684): No heartbeat from core client for 30 sec - exiting 10:50:48 (4684): No heartbeat from core client for 30 sec - exiting 10:50:49 (4684): No heartbeat from core client for 30 sec - exiting 10:50:50 (4684): No heartbeat from core client for 30 sec - exiting 10:50:51 (4684): No heartbeat from core client for 30 sec - exiting 10:50:52 (4684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:56:55 (4032): No heartbeat from core client for 30 sec - exiting 16:56:56 (4032): No heartbeat from core client for 30 sec - exiting 16:56:57 (4032): No heartbeat from core client for 30 sec - exiting 16:56:58 (4032): No heartbeat from core client for 30 sec - exiting 16:56:59 (4032): No heartbeat from core client for 30 sec - exiting 16:57:00 (4032): No heartbeat from core client for 30 sec - exiting 16:57:02 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:34:42 (4672): No heartbeat from core client for 30 sec - exiting 10:34:43 (4672): No heartbeat from core client for 30 sec - exiting 10:34:44 (4672): No heartbeat from core client for 30 sec - exiting 10:34:45 (4672): No heartbeat from core client for 30 sec - exiting 10:34:47 (4672): No heartbeat from core client for 30 sec - exiting 10:34:48 (4672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:11 (5704): No heartbeat from core client for 30 sec - exiting 16:45:12 (5704): No heartbeat from core client for 30 sec - exiting 16:45:13 (5704): No heartbeat from core client for 30 sec - exiting 16:45:14 (5704): No heartbeat from core client for 30 sec - exiting 16:45:15 (5704): No heartbeat from core client for 30 sec - exiting 16:45:17 (5704): No heartbeat from core client for 30 sec - exiting 16:45:18 (5704): No heartbeat from core client for 30 sec - exiting 16:45:19 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:53:22 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:57:56 (3388): No heartbeat from core client for 30 sec - exiting 15:57:57 (3388): No heartbeat from core client for 30 sec - exiting 15:57:58 (3388): No heartbeat from core client for 30 sec - exiting 15:57:59 (3388): No heartbeat from core client for 30 sec - exiting 15:58:00 (3388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77183AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BA71F3 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2012 19:47:32	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	259,200	455,941	1.7590
29 Oct 2012 20:58:43	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	233,280	410,593	1.7601
27 Oct 2012 16:27:09	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	207,360	365,269	1.7615
24 Oct 2012 06:16:10	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	181,440	319,596	1.7614
20 Oct 2012 20:02:30	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	155,520	274,044	1.7621
16 Oct 2012 09:04:41	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	129,600	228,506	1.7632
14 Oct 2012 14:22:09	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	103,680	183,772	1.7725
12 Oct 2012 06:55:04	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	77,760	138,512	1.7813
10 Oct 2012 12:01:50	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	51,840	92,587	1.7860
09 Oct 2012 06:30:03	1193903	15277333	hadcm3n_zmr8_1880_40_008199682_0	25,920	46,504	1.7941