Task 16014538

Name	hadcm3n_n3f2_1880_40_008374713_4
Workunit	8525572
Created	13 Sep 2013, 8:22:49 UTC
Sent	13 Sep 2013, 8:35:28 UTC
Report deadline	13 Dec 2013, 16:02:39 UTC
Received	13 Oct 2013, 10:09:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1072815
Run time	11 days 12 hours 42 min 40 sec
CPU time	11 days 1 hours 34 min 3 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.68 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 11:47:21 (4076): No heartbeat from core client for 30 sec - exiting 11:47:23 (4076): No heartbeat from core client for 30 sec - exiting 11:47:24 (4076): No heartbeat from core client for 30 sec - exiting 11:47:25 (4076): No heartbeat from core client for 30 sec - exiting 11:47:26 (4076): No heartbeat from core client for 30 sec - exiting 11:47:27 (4076): No heartbeat from core client for 30 sec - exiting 11:47:28 (4076): No heartbeat from core client for 30 sec - exiting 11:47:29 (4076): No heartbeat from core client for 30 sec - exiting 11:47:30 (4076): No heartbeat from core client for 30 sec - exiting 11:47:31 (4076): No heartbeat from core client for 30 sec - exiting 11:47:32 (4076): No heartbeat from core client for 30 sec - exiting 11:47:34 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:50:25 (2352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:17:35 (3592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... 18:40:55 (3724): No heartbeat from core client for 30 sec - exiting 18:40:56 (3724): No heartbeat from core client for 30 sec - exiting 18:40:57 (3724): No heartbeat from core client for 30 sec - exiting 18:40:58 (3724): No heartbeat from core client for 30 sec - exiting 18:40:59 (3724): No heartbeat from core client for 30 sec - exiting 18:41:00 (3724): No heartbeat from core client for 30 sec - exiting 18:41:01 (3724): No heartbeat from core client for 30 sec - exiting 18:41:02 (3724): No heartbeat from core client for 30 sec - exiting 18:41:03 (3724): No heartbeat from core client for 30 sec - exiting 18:41:04 (3724): No heartbeat from core client for 30 sec - exiting 18:41:05 (3724): No heartbeat from core client for 30 sec - exiting 18:41:06 (3724): No heartbeat from core client for 30 sec - exiting 18:41:07 (3724): No heartbeat from core client for 30 sec - exiting 18:41:08 (3724): No heartbeat from core client for 30 sec - exiting 18:41:10 (3724): No heartbeat from core client for 30 sec - exiting 18:41:11 (3724): No heartbeat from core client for 30 sec - exiting 18:41:12 (3724): No heartbeat from core client for 30 sec - exiting 18:41:13 (3724): No heartbeat from core client for 30 sec - exiting 18:41:14 (3724): No heartbeat from core client for 30 sec - exiting 18:41:15 (3724): No heartbeat from core client for 30 sec - exiting 18:41:16 (3724): No heartbeat from core client for 30 sec - exiting 18:41:17 (3724): No heartbeat from core client for 30 sec - exiting 18:41:18 (3724): No heartbeat from core client for 30 sec - exiting 18:41:19 (3724): No heartbeat from core client for 30 sec - exiting 18:41:20 (3724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:41:21 (3724): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:42:38 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4192, iMonCtr=1 Model crash detected, will try to restart... 18:56:32 (6084): No heartbeat from core client for 30 sec - exiting 18:56:33 (6084): No heartbeat from core client for 30 sec - exiting 18:56:34 (6084): No heartbeat from core client for 30 sec - exiting 18:56:35 (6084): No heartbeat from core client for 30 sec - exiting 18:56:36 (6084): No heartbeat from core client for 30 sec - exiting 18:56:37 (6084): No heartbeat from core client for 30 sec - exiting 18:56:38 (6084): No heartbeat from core client for 30 sec - exiting 18:56:39 (6084): No heartbeat from core client for 30 sec - exiting 18:56:41 (6084): No heartbeat from core client for 30 sec - exiting 18:56:42 (6084): No heartbeat from core client for 30 sec - exiting 18:56:43 (6084): No heartbeat from core client for 30 sec - exiting 18:56:44 (6084): No heartbeat from core client for 30 sec - exiting 18:56:45 (6084): No heartbeat from core client for 30 sec - exiting 18:56:46 (6084): No heartbeat from core client for 30 sec - exiting 18:56:47 (6084): No heartbeat from core client for 30 sec - exiting 18:56:48 (6084): No heartbeat from core client for 30 sec - exiting 18:56:49 (6084): No heartbeat from core client for 30 sec - exiting 18:56:50 (6084): No heartbeat from core client for 30 sec - exiting 18:56:51 (6084): No heartbeat from core client for 30 sec - exiting 18:56:53 (6084): No heartbeat from core client for 30 sec - exiting 18:56:54 (6084): No heartbeat from core client for 30 sec - exiting 18:56:55 (6084): No heartbeat from core client for 30 sec - exiting 18:56:56 (6084): No heartbeat from core client for 30 sec - exiting 18:56:57 (6084): No heartbeat from core client for 30 sec - exiting 18:56:58 (6084): No heartbeat from core client for 30 sec - exiting 18:56:59 (6084): No heartbeat from core client for 30 sec - exiting 18:57:00 (6084): No heartbeat from core client for 30 sec - exiting 18:57:01 (6084): No heartbeat from core client for 30 sec - exiting 18:57:02 (6084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5464, iMonCtr=1 Model crash detected, will try to restart... 18:20:38 (2692): No heartbeat from core client for 30 sec - exiting 18:20:40 (2692): No heartbeat from core client for 30 sec - exiting 18:20:41 (2692): No heartbeat from core client for 30 sec - exiting 18:20:42 (2692): No heartbeat from core client for 30 sec - exiting 18:20:43 (2692): No heartbeat from core client for 30 sec - exiting 18:20:44 (2692): No heartbeat from core client for 30 sec - exiting 18:20:45 (2692): No heartbeat from core client for 30 sec - exiting 18:20:46 (2692): No heartbeat from core client for 30 sec - exiting 18:20:47 (2692): No heartbeat from core client for 30 sec - exiting 18:20:48 (2692): No heartbeat from core client for 30 sec - exiting 18:20:50 (2692): No heartbeat from core client for 30 sec - exiting 18:20:51 (2692): No heartbeat from core client for 30 sec - exiting 18:20:52 (2692): No heartbeat from core client for 30 sec - exiting 18:20:53 (2692): No heartbeat from core client for 30 sec - exiting 18:20:54 (2692): No heartbeat from core client for 30 sec - exiting 18:20:55 (2692): No heartbeat from core client for 30 sec - exiting 18:20:56 (2692): No heartbeat from core client for 30 sec - exiting 18:20:57 (2692): No heartbeat from core client for 30 sec - exiting 18:20:58 (2692): No heartbeat from core client for 30 sec - exiting 18:20:59 (2692): No heartbeat from core client for 30 sec - exiting 18:21:01 (2692): No heartbeat from core client for 30 sec - exiting 18:21:02 (2692): No heartbeat from core client for 30 sec - exiting 18:21:03 (2692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:45:24 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:16:14 (6880): No heartbeat from core client for 30 sec - exiting 09:16:16 (6880): No heartbeat from core client for 30 sec - exiting 09:16:17 (6880): No heartbeat from core client for 30 sec - exiting 09:16:18 (6880): No heartbeat from core client for 30 sec - exiting 09:16:19 (6880): No heartbeat from core client for 30 sec - exiting 09:16:20 (6880): No heartbeat from core client for 30 sec - exiting 09:16:21 (6880): No heartbeat from core client for 30 sec - exiting 09:16:22 (6880): No heartbeat from core client for 30 sec - exiting 09:16:23 (6880): No heartbeat from core client for 30 sec - exiting 09:16:24 (6880): No heartbeat from core client for 30 sec - exiting 09:16:25 (6880): No heartbeat from core client for 30 sec - exiting 09:16:26 (6880): No heartbeat from core client for 30 sec - exiting 09:16:27 (6880): No heartbeat from core client for 30 sec - exiting 09:16:28 (6880): No heartbeat from core client for 30 sec - exiting 09:16:29 (6880): No heartbeat from core client for 30 sec - exiting 09:16:30 (6880): No heartbeat from core client for 30 sec - exiting 09:16:31 (6880): No heartbeat from core client for 30 sec - exiting 09:16:32 (6880): No heartbeat from core client for 30 sec - exiting 09:16:33 (6880): No heartbeat from core client for 30 sec - exiting 09:16:34 (6880): No heartbeat from core client for 30 sec - exiting 09:16:35 (6880): No heartbeat from core client for 30 sec - exiting 09:16:36 (6880): No heartbeat from core client for 30 sec - exiting 09:16:37 (6880): No heartbeat from core client for 30 sec - exiting 09:16:38 (6880): No heartbeat from core client for 30 sec - exiting 09:16:39 (6880): No heartbeat from core client for 30 sec - exiting 09:16:40 (6880): No heartbeat from core client for 30 sec - exiting 09:16:41 (6880): No heartbeat from core client for 30 sec - exiting 09:16:42 (6880): No heartbeat from core client for 30 sec - exiting 09:16:43 (6880): No heartbeat from core client for 30 sec - exiting 09:16:44 (6880): No heartbeat from core client for 30 sec - exiting 09:16:45 (6880): No heartbeat from core client for 30 sec - exiting 09:16:46 (6880): No heartbeat from core client for 30 sec - exiting 09:16:47 (6880): No heartbeat from core client for 30 sec - exiting 09:16:48 (6880): No heartbeat from core client for 30 sec - exiting 09:16:49 (6880): No heartbeat from core client for 30 sec - exiting 09:16:50 (6880): No heartbeat from core client for 30 sec - exiting 09:16:51 (6880): No heartbeat from core client for 30 sec - exiting 09:16:52 (6880): No heartbeat from core client for 30 sec - exiting 09:16:53 (6880): No heartbeat from core client for 30 sec - exiting 09:16:54 (6880): No heartbeat from core client for 30 sec - exiting 09:16:55 (6880): No heartbeat from core client for 30 sec - exiting 09:16:56 (6880): No heartbeat from core client for 30 sec - exiting 09:16:57 (6880): No heartbeat from core client for 30 sec - exiting 09:16:58 (6880): No heartbeat from core client for 30 sec - exiting 09:16:59 (6880): No heartbeat from core client for 30 sec - exiting 09:17:00 (6880): No heartbeat from core client for 30 sec - exiting 09:17:01 (6880): No heartbeat from core client for 30 sec - exiting 09:17:02 (6880): No heartbeat from core client for 30 sec - exiting 09:17:03 (6880): No heartbeat from core client for 30 sec - exiting 09:17:04 (6880): No heartbeat from core client for 30 sec - exiting 09:17:05 (6880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:17:06 (6880): No heartbeat from core client for 30 sec - exiting 09:17:07 (6880): No heartbeat from core client for 30 sec - exiting 09:17:08 (6880): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:39:05 (3700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:39:41 (4928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:56:22 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:19:14 (6968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:35:12 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2884, iMonCtr=1 Model crash detected, will try to restart... 18:44:46 (7104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:40:49 (5868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:15:00 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:25:56 (4400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:25:57 (4400): No heartbeat from core client for 30 sec - exiting 18:25:58 (4400): No heartbeat from core client for 30 sec - exiting 18:25:59 (4400): No heartbeat from core client for 30 sec - exiting 18:26:00 (4400): No heartbeat from core client for 30 sec - exiting 18:26:01 (4400): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:38:07 (2616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:22:07 (5504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:29:00 (3092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:38:28 (6948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4032, iMonCtr=1 Model crash detected, will try to restart... 15:19:05 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:38:18 (3184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:41:15 (6248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:42:39 (6964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:19:03 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:19:46 (6880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:24:37 (3692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:22:08 (5732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:15:52 (3744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:17:41 (6248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3396, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:27:26 (6400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:44:03 (5304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:44:04 (5304): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:52:10 (4684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77E97383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 00:59:56 (6708): No heartbeat from core client for 30 sec - exiting 00:59:57 (6708): No heartbeat from core client for 30 sec - exiting 00:59:58 (6708): No heartbeat from core client for 30 sec - exiting 00:59:59 (6708): No heartbeat from core client for 30 sec - exiting 01:00:00 (6708): No heartbeat from core client for 30 sec - exiting 01:00:01 (6708): No heartbeat from core client for 30 sec - exiting 01:00:02 (6708): No heartbeat from core client for 30 sec - exiting 01:00:03 (6708): No heartbeat from core client for 30 sec - exiting 01:00:04 (6708): No heartbeat from core client for 30 sec - exiting 01:00:05 (6708): No heartbeat from core client for 30 sec - exiting 01:00:06 (6708): No heartbeat from core client for 30 sec - exiting 01:00:07 (6708): No heartbeat from core client for 30 sec - exiting 01:00:08 (6708): No heartbeat from core client for 30 sec - exiting 01:00:09 (6708): No heartbeat from core client for 30 sec - exiting 01:00:10 (6708): No heartbeat from core client for 30 sec - exiting 01:00:11 (6708): No heartbeat from core client for 30 sec - exiting 01:00:12 (6708): No heartbeat from core client for 30 sec - exiting 01:00:13 (6708): No heartbeat from core client for 30 sec - exiting 01:00:14 (6708): No heartbeat from core client for 30 sec - exiting 01:00:15 (6708): No heartbeat from core client for 30 sec - exiting 01:00:16 (6708): No heartbeat from core client for 30 sec - exiting 01:00:17 (6708): No heartbeat from core client for 30 sec - exiting 01:00:18 (6708): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77E97383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 11:56:53 (5752): No heartbeat from core client for 30 sec - exiting 11:56:54 (5752): No heartbeat from core client for 30 sec - exiting 11:56:55 (5752): No heartbeat from core client for 30 sec - exiting 11:56:56 (5752): No heartbeat from core client for 30 sec - exiting 11:56:57 (5752): No heartbeat from core client for 30 sec - exiting 11:56:58 (5752): No heartbeat from core client for 30 sec - exiting 11:56:59 (5752): No heartbeat from core client for 30 sec - exiting 11:57:00 (5752): No heartbeat from core client for 30 sec - exiting 11:57:01 (5752): No heartbeat from core client for 30 sec - exiting 11:57:02 (5752): No heartbeat from core client for 30 sec - exiting 11:57:03 (5752): No heartbeat from core client for 30 sec - exiting 11:57:04 (5752): No heartbeat from core client for 30 sec - exiting 11:57:05 (5752): No heartbeat from core client for 30 sec - exiting 11:57:06 (5752): No heartbeat from core client for 30 sec - exiting 11:57:07 (5752): No heartbeat from core client for 30 sec - exiting 11:57:08 (5752): No heartbeat from core client for 30 sec - exiting 11:57:09 (5752): No heartbeat from core client for 30 sec - exiting 11:57:10 (5752): No heartbeat from core client for 30 sec - exiting 11:57:11 (5752): No heartbeat from core client for 30 sec - exiting 11:57:12 (5752): No heartbeat from core client for 30 sec - exiting 11:57:13 (5752): No heartbeat from core client for 30 sec - exiting 11:57:14 (5752): No heartbeat from core client for 30 sec - exiting 11:57:15 (5752): No heartbeat from core client for 30 sec - exiting 11:57:16 (5752): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B43770 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B47203 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Oct 2013 19:58:42	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	518,400	944,933	1.8228
11 Oct 2013 22:44:41	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	492,480	897,815	1.8230
11 Oct 2013 10:22:11	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	466,560	851,679	1.8254
09 Oct 2013 18:27:26	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	440,640	804,051	1.8247
07 Oct 2013 18:24:24	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	414,720	758,345	1.8286
06 Oct 2013 10:36:06	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	388,800	712,516	1.8326
05 Oct 2013 11:24:51	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	362,880	666,295	1.8361
04 Oct 2013 11:53:12	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	336,960	619,628	1.8389
02 Oct 2013 23:11:31	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	311,040	573,101	1.8425
01 Oct 2013 07:15:01	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	285,120	528,024	1.8519
29 Sep 2013 10:34:29	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	259,200	482,696	1.8623
28 Sep 2013 10:50:41	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	233,280	435,458	1.8667
25 Sep 2013 20:12:15	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	207,360	386,571	1.8643
23 Sep 2013 17:41:40	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	181,440	338,491	1.8656
21 Sep 2013 14:27:22	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	155,520	290,099	1.8653
20 Sep 2013 10:24:43	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	129,600	241,867	1.8663
18 Sep 2013 19:18:21	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	103,680	191,045	1.8426
16 Sep 2013 18:49:36	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	77,760	143,679	1.8477
14 Sep 2013 21:56:14	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	51,840	96,047	1.8528
13 Sep 2013 22:49:20	1072815	16014538	hadcm3n_n3f2_1880_40_008374713_4	25,920	48,788	1.8823