Task 13328183

Name	hadcm3n_t2cz_1940_40_007435950_2
Workunit	7633453
Created	2 Sep 2011, 3:54:13 UTC
Sent	2 Sep 2011, 3:59:59 UTC
Report deadline	2 Dec 2011, 11:27:10 UTC
Received	5 Oct 2011, 5:20:19 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1124341
Run time	9 days 13 hours 10 min 34 sec
CPU time	8 days 22 hours 55 min 30 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.55 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:20:38 (4512): No heartbeat from core client for 30 sec - exiting 07:20:39 (4512): No heartbeat from core client for 30 sec - exiting 07:20:40 (4512): No heartbeat from core client for 30 sec - exiting 07:20:41 (4512): No heartbeat from core client for 30 sec - exiting 07:20:42 (4512): No heartbeat from core client for 30 sec - exiting 07:20:43 (4512): No heartbeat from core client for 30 sec - exiting 07:20:44 (4512): No heartbeat from core client for 30 sec - exiting 07:20:45 (4512): No heartbeat from core client for 30 sec - exiting 07:20:46 (4512): No heartbeat from core client for 30 sec - exiting 07:20:48 (4512): No heartbeat from core client for 30 sec - exiting 07:20:49 (4512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:58:44 (4416): No heartbeat from core client for 30 sec - exiting 14:58:45 (4416): No heartbeat from core client for 30 sec - exiting 14:58:46 (4416): No heartbeat from core client for 30 sec - exiting 14:58:47 (4416): No heartbeat from core client for 30 sec - exiting 14:58:48 (4416): No heartbeat from core client for 30 sec - exiting 14:58:50 (4416): No heartbeat from core client for 30 sec - exiting 14:58:51 (4416): No heartbeat from core client for 30 sec - exiting 14:58:52 (4416): No heartbeat from core client for 30 sec - exiting 14:58:53 (4416): No heartbeat from core client for 30 sec - exiting 14:58:54 (4416): No heartbeat from core client for 30 sec - exiting 14:58:55 (4416): No heartbeat from core client for 30 sec - exiting 14:58:56 (4416): No heartbeat from core client for 30 sec - exiting 14:58:57 (4416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:25:46 (4580): No heartbeat from core client for 30 sec - exiting 07:25:47 (4580): No heartbeat from core client for 30 sec - exiting 07:25:48 (4580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:49 (4644): No heartbeat from core client for 30 sec - exiting 13:23:51 (4644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:48:51 (4560): No heartbeat from core client for 30 sec - exiting 06:48:52 (4560): No heartbeat from core client for 30 sec - exiting 06:48:53 (4560): No heartbeat from core client for 30 sec - exiting 06:48:54 (4560): No heartbeat from core client for 30 sec - exiting 06:48:55 (4560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:55:04 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:55:05 (4344): No heartbeat from core client for 30 sec - exiting 06:22:39 (4528): No heartbeat from core client for 30 sec - exiting 06:22:40 (4528): No heartbeat from core client for 30 sec - exiting 06:22:41 (4528): No heartbeat from core client for 30 sec - exiting 06:22:42 (4528): No heartbeat from core client for 30 sec - exiting 06:22:43 (4528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3848, iMonCtr=1 Model crash detected, will try to restart... 15:17:12 (4552): No heartbeat from core client for 30 sec - exiting 15:17:13 (4552): No heartbeat from core client for 30 sec - exiting 15:17:14 (4552): No heartbeat from core client for 30 sec - exiting 15:17:15 (4552): No heartbeat from core client for 30 sec - exiting 15:17:16 (4552): No heartbeat from core client for 30 sec - exiting 15:17:18 (4552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:17:42 (4584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:31:38 (4412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:27:01 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:31:27 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:16:40 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:46:19 (4392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:13 (4508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:34:07 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:34:08 (4452): No heartbeat from core client for 30 sec - exiting 14:38:50 (4344): No heartbeat from core client for 30 sec - exiting 14:38:52 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:40:08 (4396): No heartbeat from core client for 30 sec - exiting 06:40:10 (4396): No heartbeat from core client for 30 sec - exiting 06:40:11 (4396): No heartbeat from core client for 30 sec - exiting 06:40:12 (4396): No heartbeat from core client for 30 sec - exiting 06:40:13 (4396): No heartbeat from core client for 30 sec - exiting 06:40:14 (4396): No heartbeat from core client for 30 sec - exiting 06:40:15 (4396): No heartbeat from core client for 30 sec - exiting 06:40:16 (4396): No heartbeat from core client for 30 sec - exiting 06:40:17 (4396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:27:00 (4600): No heartbeat from core client for 30 sec - exiting 17:27:01 (4600): No heartbeat from core client for 30 sec - exiting 17:27:02 (4600): No heartbeat from core client for 30 sec - exiting 17:27:03 (4600): No heartbeat from core client for 30 sec - exiting 17:27:04 (4600): No heartbeat from core client for 30 sec - exiting 17:27:06 (4600): No heartbeat from core client for 30 sec - exiting 17:27:07 (4600): No heartbeat from core client for 30 sec - exiting 17:27:08 (4600): No heartbeat from core client for 30 sec - exiting 17:27:09 (4600): No heartbeat from core client for 30 sec - exiting 17:27:10 (4600): No heartbeat from core client for 30 sec - exiting 17:27:11 (4600): No heartbeat from core client for 30 sec - exiting 17:27:12 (4600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:58:46 (4536): No heartbeat from core client for 30 sec - exiting 00:58:47 (4536): No heartbeat from core client for 30 sec - exiting 00:58:48 (4536): No heartbeat from core client for 30 sec - exiting 00:58:50 (4536): No heartbeat from core client for 30 sec - exiting 00:58:51 (4536): No heartbeat from core client for 30 sec - exiting 00:58:52 (4536): No heartbeat from core client for 30 sec - exiting 00:58:53 (4536): No heartbeat from core client for 30 sec - exiting 00:58:54 (4536): No heartbeat from core client for 30 sec - exiting 00:58:55 (4536): No heartbeat from core client for 30 sec - exiting 00:58:56 (4536): No heartbeat from core client for 30 sec - exiting 00:58:57 (4536): No heartbeat from core client for 30 sec - exiting 00:58:58 (4536): No heartbeat from core client for 30 sec - exiting 00:58:59 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:06:56 (4360): No heartbeat from core client for 30 sec - exiting 09:06:57 (4360): No heartbeat from core client for 30 sec - exiting 09:06:58 (4360): No heartbeat from core client for 30 sec - exiting 09:06:59 (4360): No heartbeat from core client for 30 sec - exiting 09:07:00 (4360): No heartbeat from core client for 30 sec - exiting 09:07:01 (4360): No heartbeat from core client for 30 sec - exiting 09:07:03 (4360): No heartbeat from core client for 30 sec - exiting 09:07:04 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:28 (4252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:58:03 (5032): No heartbeat from core client for 30 sec - exiting 07:58:04 (5032): No heartbeat from core client for 30 sec - exiting 07:58:05 (5032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:54:36 (4344): No heartbeat from core client for 30 sec - exiting 00:54:37 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:36:30 (4436): No heartbeat from core client for 30 sec - exiting 09:36:31 (4436): No heartbeat from core client for 30 sec - exiting 09:36:32 (4436): No heartbeat from core client for 30 sec - exiting 09:36:33 (4436): No heartbeat from core client for 30 sec - exiting 09:36:34 (4436): No heartbeat from core client for 30 sec - exiting 09:36:35 (4436): No heartbeat from core client for 30 sec - exiting 09:36:37 (4436): No heartbeat from core client for 30 sec - exiting 09:36:38 (4436): No heartbeat from core client for 30 sec - exiting 09:36:39 (4436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:44:47 (4188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:44:55 (4152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:58:20 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:06:58 (2832): No heartbeat from core client for 30 sec - exiting 19:06:59 (2832): No heartbeat from core client for 30 sec - exiting 19:07:01 (2832): No heartbeat from core client for 30 sec - exiting 19:07:02 (2832): No heartbeat from core client for 30 sec - exiting 19:07:03 (2832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:05:06 (4948): No heartbeat from core client for 30 sec - exiting 01:05:07 (4948): No heartbeat from core client for 30 sec - exiting 01:05:08 (4948): No heartbeat from core client for 30 sec - exiting 01:05:09 (4948): No heartbeat from core client for 30 sec - exiting 01:05:10 (4948): No heartbeat from core client for 30 sec - exiting 01:05:11 (4948): No heartbeat from core client for 30 sec - exiting 01:05:12 (4948): No heartbeat from core client for 30 sec - exiting 01:05:13 (4948): No heartbeat from core client for 30 sec - exiting 01:05:14 (4948): No heartbeat from core client for 30 sec - exiting 01:05:15 (4948): No heartbeat from core client for 30 sec - exiting 01:05:16 (4948): No heartbeat from core client for 30 sec - exiting 01:05:17 (4948): No heartbeat from core client for 30 sec - exiting 01:05:18 (4948): No heartbeat from core client for 30 sec - exiting 01:05:19 (4948): No heartbeat from core client for 30 sec - exiting 01:05:20 (4948): No heartbeat from core client for 30 sec - exiting 01:05:21 (4948): No heartbeat from core client for 30 sec - exiting 01:05:22 (4948): No heartbeat from core client for 30 sec - exiting 01:05:23 (4948): No heartbeat from core client for 30 sec - exiting 01:05:24 (4948): No heartbeat from core client for 30 sec - exiting 01:05:25 (4948): No heartbeat from core client for 30 sec - exiting 01:05:26 (4948): No heartbeat from core client for 30 sec - exiting 01:05:27 (4948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:25:37 (4548): No heartbeat from core client for 30 sec - exiting 08:25:38 (4548): No heartbeat from core client for 30 sec - exiting 08:25:39 (4548): No heartbeat from core client for 30 sec - exiting 08:25:40 (4548): No heartbeat from core client for 30 sec - exiting 08:25:41 (4548): No heartbeat from core client for 30 sec - exiting 08:25:42 (4548): No heartbeat from core client for 30 sec - exiting 08:25:43 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:08:06 (4504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4344, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4640, iMonCtr=1 Model crash detected, will try to restart... 17:35:58 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:12 (4288): No heartbeat from core client for 30 sec - exiting 02:02:13 (4288): No heartbeat from core client for 30 sec - exiting 02:02:14 (4288): No heartbeat from core client for 30 sec - exiting 02:02:16 (4288): No heartbeat from core client for 30 sec - exiting 02:02:17 (4288): No heartbeat from core client for 30 sec - exiting 02:02:18 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:34:19 (3712): No heartbeat from core client for 30 sec - exiting 02:34:21 (3712): No heartbeat from core client for 30 sec - exiting 02:34:22 (3712): No heartbeat from core client for 30 sec - exiting 02:34:23 (3712): No heartbeat from core client for 30 sec - exiting 02:34:24 (3712): No heartbeat from core client for 30 sec - exiting 02:34:25 (3712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:34:26 (3712): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5516, iMonCtr=1 Model crash detected, will try to restart... 08:45:51 (4424): No heartbeat from core client for 30 sec - exiting 08:45:53 (4424): No heartbeat from core client for 30 sec - exiting 08:45:54 (4424): No heartbeat from core client for 30 sec - exiting 08:45:55 (4424): No heartbeat from core client for 30 sec - exiting 08:45:56 (4424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:48:36 (4028): No heartbeat from core client for 30 sec - exiting 15:48:37 (4028): No heartbeat from core client for 30 sec - exiting 15:48:38 (4028): No heartbeat from core client for 30 sec - exiting 15:48:39 (4028): No heartbeat from core client for 30 sec - exiting 15:48:40 (4028): No heartbeat from core client for 30 sec - exiting 15:48:41 (4028): No heartbeat from core client for 30 sec - exiting 15:48:42 (4028): No heartbeat from core client for 30 sec - exiting 15:48:44 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... 06:40:33 (4384): No heartbeat from core client for 30 sec - exiting 06:40:35 (4384): No heartbeat from core client for 30 sec - exiting 06:40:36 (4384): No heartbeat from core client for 30 sec - exiting 06:40:37 (4384): No heartbeat from core client for 30 sec - exiting 06:40:38 (4384): No heartbeat from core client for 30 sec - exiting 06:40:39 (4384): No heartbeat from core client for 30 sec - exiting 06:40:40 (4384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:48 (4296): No heartbeat from core client for 30 sec - exiting 16:04:49 (4296): No heartbeat from core client for 30 sec - exiting 16:04:50 (4296): No heartbeat from core client for 30 sec - exiting 16:04:51 (4296): No heartbeat from core client for 30 sec - exiting 16:04:52 (4296): No heartbeat from core client for 30 sec - exiting 16:04:53 (4296): No heartbeat from core client for 30 sec - exiting 16:04:54 (4296): No heartbeat from core client for 30 sec - exiting 16:04:55 (4296): No heartbeat from core client for 30 sec - exiting 16:04:56 (4296): No heartbeat from core client for 30 sec - exiting 16:04:58 (4296): No heartbeat from core client for 30 sec - exiting 16:04:59 (4296): No heartbeat from core client for 30 sec - exiting 16:05:00 (4296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:01:33 (4340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:32 (4380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:32:53 (4268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:42:38 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:37 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:25:58 (4156): No heartbeat from core client for 30 sec - exiting 18:25:59 (4156): No heartbeat from core client for 30 sec - exiting 18:26:00 (4156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:55:37 (1636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:05:50 (1740): No heartbeat from core client for 30 sec - exiting 23:05:52 (1740): No heartbeat from core client for 30 sec - exiting 23:05:53 (1740): No heartbeat from core client for 30 sec - exiting 23:05:54 (1740): No heartbeat from core client for 30 sec - exiting 23:05:55 (1740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:29:23 (3692): No heartbeat from core client for 30 sec - exiting 08:29:24 (3692): No heartbeat from core client for 30 sec - exiting 08:29:25 (3692): No heartbeat from core client for 30 sec - exiting 08:29:26 (3692): No heartbeat from core client for 30 sec - exiting 08:29:27 (3692): No heartbeat from core client for 30 sec - exiting 08:29:29 (3692): No heartbeat from core client for 30 sec - exiting 08:29:30 (3692): No heartbeat from core client for 30 sec - exiting 08:29:31 (3692): No heartbeat from core client for 30 sec - exiting 08:29:32 (3692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3816, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4112, iMonCtr=1 Model crash detected, will try to restart... 08:35:53 (3892): No heartbeat from core client for 30 sec - exiting 08:35:54 (3892): No heartbeat from core client for 30 sec - exiting 08:35:55 (3892): No heartbeat from core client for 30 sec - exiting 08:35:56 (3892): No heartbeat from core client for 30 sec - exiting 08:35:57 (3892): No heartbeat from core client for 30 sec - exiting 08:35:58 (3892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:40:26 (2072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:14:03 (2652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:11:40 (1960): No heartbeat from core client for 30 sec - exiting 22:11:41 (1960): No heartbeat from core client for 30 sec - exiting 22:11:42 (1960): No heartbeat from core client for 30 sec - exiting 22:11:43 (1960): No heartbeat from core client for 30 sec - exiting 22:11:44 (1960): No heartbeat from core client for 30 sec - exiting 22:11:46 (1960): No heartbeat from core client for 30 sec - exiting 22:11:47 (1960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:08:43 (1280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:04:55 (3480): No heartbeat from core client for 30 sec - exiting 14:04:57 (3480): No heartbeat from core client for 30 sec - exiting 14:04:58 (3480): No heartbeat from core client for 30 sec - exiting 14:04:59 (3480): No heartbeat from core client for 30 sec - exiting 14:05:00 (3480): No heartbeat from core client for 30 sec - exiting 14:05:01 (3480): No heartbeat from core client for 30 sec - exiting 14:05:02 (3480): No heartbeat from core client for 30 sec - exiting 14:05:03 (3480): No heartbeat from core client for 30 sec - exiting 14:05:04 (3480): No heartbeat from core client for 30 sec - exiting 14:05:05 (3480): No heartbeat from core client for 30 sec - exiting 14:05:06 (3480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:42 (2148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:36:26 (2264): No heartbeat from core client for 30 sec - exiting 15:36:27 (2264): No heartbeat from core client for 30 sec - exiting 15:36:28 (2264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BA3A93 read attempt to address 0x409E0EE6 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Oct 2011 13:41:19	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	518,400	773,708	1.4925
02 Oct 2011 07:14:28	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	492,480	735,639	1.4937
30 Sep 2011 12:49:51	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	466,560	697,151	1.4942
27 Sep 2011 18:40:59	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	440,640	659,807	1.4974
25 Sep 2011 20:15:33	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	414,720	622,030	1.4999
23 Sep 2011 18:30:19	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	388,800	583,050	1.4996
21 Sep 2011 19:17:46	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	362,880	542,889	1.4961
19 Sep 2011 17:46:05	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	336,960	503,956	1.4956
17 Sep 2011 15:38:44	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	311,040	466,530	1.4999
14 Sep 2011 08:00:34	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	285,120	427,714	1.5001
13 Sep 2011 06:11:55	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	259,200	388,782	1.4999
12 Sep 2011 18:30:58	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	233,280	347,559	1.4899
11 Sep 2011 18:03:41	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	207,360	307,529	1.4831
10 Sep 2011 21:12:35	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	181,440	268,603	1.4804
09 Sep 2011 17:12:53	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	155,520	229,242	1.4740
07 Sep 2011 12:16:38	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	129,600	191,010	1.4738
05 Sep 2011 20:14:35	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	103,680	153,300	1.4786
04 Sep 2011 16:34:29	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	77,760	115,470	1.4850
03 Sep 2011 23:10:21	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	51,840	77,396	1.4930
03 Sep 2011 09:52:22	1124341	13328183	hadcm3n_t2cz_1940_40_007435950_2	25,920	38,847	1.4987