Task 13287967

Name	hadcm3n_p2nb_1940_40_007420120_1
Workunit	7617755
Created	24 Aug 2011, 21:21:13 UTC
Sent	24 Aug 2011, 21:21:16 UTC
Report deadline	24 Nov 2011, 4:48:27 UTC
Received	20 Jan 2012, 16:51:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1165323
Run time	36 days 9 hours 13 min 50 sec
CPU time	35 days 6 hours 20 min 31 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.69 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:10:27 (4836): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4628, iMonCtr=1 Model crash detected, will try to restart... 10:04:52 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:44:46 (4996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:47:58 (4784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:20:38 (4484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:19:50 (4460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:53 (4880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:37:57 (824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:46:37 (4928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:09:34 (5044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:12:41 (4808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:37:45 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:03 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:47 (4616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:44:39 (4616): No heartbeat from core client for 30 sec - exiting 09:56:09 (4716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:54:23 (5028): No heartbeat from core client for 30 sec - exiting 09:54:24 (5028): No heartbeat from core client for 30 sec - exiting 09:54:25 (5028): No heartbeat from core client for 30 sec - exiting 09:54:26 (5028): No heartbeat from core client for 30 sec - exiting 09:54:28 (5028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:55:02 (5056): No heartbeat from core client for 30 sec - exiting 09:55:04 (5056): No heartbeat from core client for 30 sec - exiting 09:55:05 (5056): No heartbeat from core client for 30 sec - exiting 09:55:06 (5056): No heartbeat from core client for 30 sec - exiting 09:55:07 (5056): No heartbeat from core client for 30 sec - exiting 09:55:08 (5056): No heartbeat from core client for 30 sec - exiting 09:55:09 (5056): No heartbeat from core client for 30 sec - exiting 09:55:10 (5056): No heartbeat from core client for 30 sec - exiting 09:55:11 (5056): No heartbeat from core client for 30 sec - exiting 09:55:12 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:19:49 (4684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:18:34 (4588): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:39:48 (6652): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:39:49 (6652): No heartbeat from core client for 30 sec - exiting 15:09:36 (5056): No heartbeat from core client for 30 sec - exiting 15:09:38 (5056): No heartbeat from core client for 30 sec - exiting 15:09:39 (5056): No heartbeat from core client for 30 sec - exiting 15:09:40 (5056): No heartbeat from core client for 30 sec - exiting 15:09:41 (5056): No heartbeat from core client for 30 sec - exiting 15:09:42 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:43 (5056): No heartbeat from core client for 30 sec - exiting 13:04:09 (7924): No heartbeat from core client for 30 sec - exiting 13:04:11 (7924): No heartbeat from core client for 30 sec - exiting 13:04:12 (7924): No heartbeat from core client for 30 sec - exiting 13:04:13 (7924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:04:52 (5776): No heartbeat from core client for 30 sec - exiting 13:04:53 (5776): No heartbeat from core client for 30 sec - exiting 13:04:54 (5776): No heartbeat from core client for 30 sec - exiting 13:04:55 (5776): No heartbeat from core client for 30 sec - exiting 13:04:56 (5776): No heartbeat from core client for 30 sec - exiting 13:04:57 (5776): No heartbeat from core client for 30 sec - exiting 13:04:58 (5776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:53 (4356): No heartbeat from core client for 30 sec - exiting 10:30:54 (4356): No heartbeat from core client for 30 sec - exiting 10:30:55 (4356): No heartbeat from core client for 30 sec - exiting 10:30:56 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:40:43 (4688): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 17:40:46 (4688): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 10:59:54 (4528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:59:55 (4528): No heartbeat from core client for 30 sec - exiting 10:59:56 (4528): No heartbeat from core client for 30 sec - exiting 10:59:57 (4528): No heartbeat from core client for 30 sec - exiting 10:59:59 (4528): No heartbeat from core client for 30 sec - exiting 11:00:00 (4528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 09:06:22 (4128): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:22:55 (1468): No heartbeat from core client for 30 sec - exiting 10:22:58 (1468): No heartbeat from core client for 30 sec - exiting 10:22:59 (1468): No heartbeat from core client for 30 sec - exiting 10:23:00 (1468): No heartbeat from core client for 30 sec - exiting 10:23:01 (1468): No heartbeat from core client for 30 sec - exiting 10:23:02 (1468): No heartbeat from core client for 30 sec - exiting 10:23:03 (1468): No heartbeat from core client for 30 sec - exiting 10:23:04 (1468): No heartbeat from core client for 30 sec - exiting 10:23:05 (1468): No heartbeat from core client for 30 sec - exiting 10:23:06 (1468): No heartbeat from core client for 30 sec - exiting 10:23:08 (1468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:07 (5116): No heartbeat from core client for 30 sec - exiting 09:55:08 (5116): No heartbeat from core client for 30 sec - exiting 09:55:09 (5116): No heartbeat from core client for 30 sec - exiting 09:55:10 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:51 (4660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5428, iMonCtr=1 Model crash detected, will try to restart... 08:11:32 (1592): No heartbeat from core client for 30 sec - exiting 08:11:33 (1592): No heartbeat from core client for 30 sec - exiting 08:11:34 (1592): No heartbeat from core client for 30 sec - exiting 08:11:35 (1592): No heartbeat from core client for 30 sec - exiting 08:11:37 (1592): No heartbeat from core client for 30 sec - exiting 08:11:38 (1592): No heartbeat from core client for 30 sec - exiting 08:11:39 (1592): No heartbeat from core client for 30 sec - exiting 08:11:40 (1592): No heartbeat from core client for 30 sec - exiting 08:11:41 (1592): No heartbeat from core client for 30 sec - exiting 08:11:42 (1592): No heartbeat from core client for 30 sec - exiting 08:11:43 (1592): No heartbeat from core client for 30 sec - exiting 08:11:44 (1592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:41:17 (612): No heartbeat from core client for 30 sec - exiting 09:41:19 (612): No heartbeat from core client for 30 sec - exiting 09:41:20 (612): No heartbeat from core client for 30 sec - exiting 09:41:21 (612): No heartbeat from core client for 30 sec - exiting 09:41:22 (612): No heartbeat from core client for 30 sec - exiting 09:41:23 (612): No heartbeat from core client for 30 sec - exiting 09:41:24 (612): No heartbeat from core client for 30 sec - exiting 09:41:25 (612): No heartbeat from core client for 30 sec - exiting 09:41:26 (612): No heartbeat from core client for 30 sec - exiting 09:41:27 (612): No heartbeat from core client for 30 sec - exiting 09:41:28 (612): No heartbeat from core client for 30 sec - exiting 09:41:30 (612): No heartbeat from core client for 30 sec - exiting 09:41:31 (612): No heartbeat from core client for 30 sec - exiting 09:41:32 (612): No heartbeat from core client for 30 sec - exiting 09:41:33 (612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:57:06 (4532): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:57:07 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:57:17 (5920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:03 (4904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:57 (1312): No heartbeat from core client for 30 sec - exiting 09:33:59 (1312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:43:57 (4220): No heartbeat from core client for 30 sec - exiting 10:43:58 (4220): No heartbeat from core client for 30 sec - exiting 10:43:59 (4220): No heartbeat from core client for 30 sec - exiting 10:44:00 (4220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:46:03 (1684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:37:14 (3628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:02 (960): No heartbeat from core client for 30 sec - exiting 09:43:03 (960): No heartbeat from core client for 30 sec - exiting 09:43:04 (960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:49 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:12:08 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:58:56 (1228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:56:42 (2628): No heartbeat from core client for 30 sec - exiting 09:56:43 (2628): No heartbeat from core client for 30 sec - exiting 09:56:44 (2628): No heartbeat from core client for 30 sec - exiting 09:56:45 (2628): No heartbeat from core client for 30 sec - exiting 09:56:46 (2628): No heartbeat from core client for 30 sec - exiting 09:56:47 (2628): No heartbeat from core client for 30 sec - exiting 09:56:49 (2628): No heartbeat from core client for 30 sec - exiting 09:56:50 (2628): No heartbeat from core client for 30 sec - exiting 09:56:51 (2628): No heartbeat from core client for 30 sec - exiting 09:56:52 (2628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:45 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:37:57 (888): No heartbeat from core client for 30 sec - exiting 16:37:59 (888): No heartbeat from core client for 30 sec - exiting 16:38:00 (888): No heartbeat from core client for 30 sec - exiting 16:38:01 (888): No heartbeat from core client for 30 sec - exiting 16:38:02 (888): No heartbeat from core client for 30 sec - exiting 16:38:03 (888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:24:58 (4668): No heartbeat from core client for 30 sec - exiting 09:25:00 (4668): No heartbeat from core client for 30 sec - exiting 09:25:01 (4668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:02 (4668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 08:51:15 (4056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:44:42 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:03:03 (1836): No heartbeat from core client for 30 sec - exiting 09:03:04 (1836): No heartbeat from core client for 30 sec - exiting 09:03:05 (1836): No heartbeat from core client for 30 sec - exiting 09:03:06 (1836): No heartbeat from core client for 30 sec - exiting 09:03:07 (1836): No heartbeat from core client for 30 sec - exiting 09:03:08 (1836): No heartbeat from core client for 30 sec - exiting 09:03:09 (1836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:03:02 (4500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1952, iMonCtr=1 Model crash detected, will try to restart... 09:10:50 (4580): No heartbeat from core client for 30 sec - exiting 09:10:52 (4580): No heartbeat from core client for 30 sec - exiting 09:10:53 (4580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:54 (4580): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:50:21 (1552): No heartbeat from core client for 30 sec - exiting 09:50:22 (1552): No heartbeat from core client for 30 sec - exiting 09:50:23 (1552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:50:24 (1552): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 00:08:55 (1164): No heartbeat from core client for 30 sec - exiting 00:08:56 (1164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:38:43 (4968): No heartbeat from core client for 30 sec - exiting 22:38:44 (4968): No heartbeat from core client for 30 sec - exiting 22:38:45 (4968): No heartbeat from core client for 30 sec - exiting 22:38:46 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:39:18 (3752): No heartbeat from core client for 30 sec - exiting 12:39:19 (3752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C09:45:46 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:13:09 (1276): No heartbeat from core client for 30 sec - exiting 10:13:11 (1276): No heartbeat from core client for 30 sec - exiting 10:13:12 (1276): No heartbeat from core client for 30 sec - exiting 10:13:13 (1276): No heartbeat from core client for 30 sec - exiting 10:13:14 (1276): No heartbeat from core client for 30 sec - exiting 10:13:15 (1276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:32:01 (3792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:28:03 (5372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:30:12 (1104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:01:12 (2676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:52 (1376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:03:09 (4124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:25:17 (1300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:48:19 (336): No heartbeat from core client for 30 sec - exiting 09:48:20 (336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:41 (1036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:35 (5124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 11:06:22 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:11:28 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:17:03 (5076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:28:23 (5612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:39:25 (4856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:22:12 (692): No heartbeat from core client for 30 sec - exiting 14:22:14 (692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:59:49 (4800): No heartbeat from core client for 30 sec - exiting 18:59:51 (4800): No heartbeat from core client for 30 sec - exiting 18:59:52 (4800): No heartbeat from core client for 30 sec - exiting 18:59:53 (4800): No heartbeat from core client for 30 sec - exiting 18:59:54 (4800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:59:55 (4800): No heartbeat from core client for 30 sec - exiting 19:00:57 (5352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:07:05 (4568): No heartbeat from core client for 30 sec - exiting 09:07:06 (4568): No heartbeat from core client for 30 sec - exiting 09:07:07 (4568): No heartbeat from core client for 30 sec - exiting 09:07:08 (4568): No heartbeat from core client for 30 sec - exiting 09:07:09 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:45 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:15:28 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:15:30 (4548): No heartbeat from core client for 30 sec - exiting 09:16:21 (4316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:17:04 (2128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:17:42 (2756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:18:33 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:32:33 (2292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:58:20 (2080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:58:21 (2080): No heartbeat from core client for 30 sec - exiting 13:59:17 (1740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:01:04 (4392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:44 (1200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:48 (1268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:03:30 (5004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:10:49 (5560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:51:11 (4716): No heartbeat from core client for 30 sec - exiting 09:51:12 (4716): No heartbeat from core client for 30 sec - exiting 09:51:13 (4716): No heartbeat from core client for 30 sec - exiting 09:51:14 (4716): No heartbeat from core client for 30 sec - exiting 09:51:15 (4716): No heartbeat from core client for 30 sec - exiting 09:51:16 (4716): No heartbeat from core client for 30 sec - exiting 09:51:17 (4716): No heartbeat from core client for 30 sec - exiting 09:51:19 (4716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:09 (4324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:55:29 (2516): No heartbeat from core client for 30 sec - exiting 06:55:30 (2516): No heartbeat from core client for 30 sec - exiting 06:55:31 (2516): No heartbeat from core client for 30 sec - exiting 06:55:32 (2516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:10:08 (4576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:10:56 (3712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77723AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Jan 2012 15:51:30	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	1,036,800	3,045,856	2.9377
12 Jan 2012 15:41:09	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	1,010,880	2,969,392	2.9374
09 Jan 2012 10:53:53	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	984,960	2,893,480	2.9377
04 Jan 2012 04:15:03	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	959,040	2,817,762	2.9381
21 Dec 2011 15:44:41	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	933,120	2,742,067	2.9386
20 Dec 2011 14:50:47	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	907,200	2,666,708	2.9395
15 Dec 2011 18:55:45	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	881,280	2,591,413	2.9405
14 Dec 2011 21:25:51	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	855,360	2,516,261	2.9418
13 Dec 2011 17:48:59	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	829,440	2,439,922	2.9416
09 Dec 2011 01:00:46	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	803,520	2,362,735	2.9405
05 Dec 2011 15:23:05	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	777,600	2,286,698	2.9407
28 Nov 2011 15:22:22	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	751,680	2,209,295	2.9391
23 Nov 2011 10:47:34	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	725,760	2,131,845	2.9374
19 Nov 2011 04:42:37	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	699,840	2,054,938	2.9363
17 Nov 2011 10:48:54	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	673,920	1,977,649	2.9345
15 Nov 2011 17:58:54	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	648,000	1,902,193	2.9355
15 Nov 2011 17:58:54	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	622,080	1,826,829	2.9366
10 Nov 2011 01:54:20	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	596,160	1,750,031	2.9355
06 Nov 2011 07:32:52	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	570,240	1,674,022	2.9356
05 Nov 2011 10:57:40	1165323	13287967	hadcm3n_p2nb_1940_40_007420120_1	544,320	1,597,552	2.9350