Task 15896387

Name	hadcm3n_3l1k_1980_40_008281360_3
Workunit	8432495
Created	18 Jul 2013, 17:25:18 UTC
Sent	18 Jul 2013, 17:35:30 UTC
Report deadline	18 Oct 2013, 1:02:41 UTC
Received	23 Oct 2013, 7:59:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1241522
Run time	15 days 19 hours 25 min 7 sec
CPU time	15 days 3 hours 50 min 30 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:04:06 (3588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:10 (3588): No heartbeat from core client for 30 sec - exiting 15:04:11 (3588): No heartbeat from core client for 30 sec - exiting 15:04:12 (3588): No heartbeat from core client for 30 sec - exiting 15:04:13 (3588): No heartbeat from core client for 30 sec - exiting 15:04:15 (3588): No heartbeat from core client for 30 sec - exiting 15:04:16 (3588): No heartbeat from core client for 30 sec - exiting 15:04:17 (3588): No heartbeat from core client for 30 sec - exiting 15:04:18 (3588): No heartbeat from core client for 30 sec - exiting 15:04:19 (3588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:25:13 (4104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:17 (4800): No heartbeat from core client for 30 sec - exiting 09:24:18 (4800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:19 (4800): No heartbeat from core client for 30 sec - exiting 09:24:20 (4800): No heartbeat from core client for 30 sec - exiting 09:24:21 (4800): No heartbeat from core client for 30 sec - exiting 09:24:22 (4800): No heartbeat from core client for 30 sec - exiting 09:24:23 (4800): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:31:34 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:48 (4172): No heartbeat from core client for 30 sec - exiting 09:25:49 (4172): No heartbeat from core client for 30 sec - exiting 09:25:50 (4172): No heartbeat from core client for 30 sec - exiting 09:25:51 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:52 (4172): No heartbeat from core client for 30 sec - exiting 09:25:53 (4172): No heartbeat from core client for 30 sec - exiting 09:25:54 (4172): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:25:43 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:25:26 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:23:08 (1136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4448, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:18:30 (4596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4676, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:14:38 (4732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:30:40 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:30:41 (4356): No heartbeat from core client for 30 sec - exiting 09:30:42 (4356): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:40:57 (4236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77173AC3 read attempt to address 0x409219A9 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77773AC3 read attempt to address 0x409219A9 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Oct 2013 11:18:46	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	1,036,800	1,309,439	1.2630
21 Oct 2013 13:26:44	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	1,010,880	1,283,847	1.2700
18 Oct 2013 13:48:03	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	984,960	1,256,272	1.2755
17 Oct 2013 15:20:00	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	959,040	1,228,021	1.2805
16 Oct 2013 15:28:25	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	933,120	1,197,013	1.2828
15 Oct 2013 14:15:58	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	907,200	1,161,240	1.2800
14 Oct 2013 14:00:47	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	881,280	1,127,571	1.2795
11 Oct 2013 12:12:31	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	855,360	1,092,759	1.2775
11 Oct 2013 02:21:50	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	829,440	1,059,295	1.2771
10 Oct 2013 16:20:30	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	803,520	1,024,935	1.2756
09 Oct 2013 11:59:33	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	777,600	991,418	1.2750
08 Oct 2013 13:25:05	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	751,680	962,993	1.2811
07 Oct 2013 13:55:44	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	725,760	933,233	1.2859
04 Oct 2013 08:00:54	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	699,840	900,412	1.2866
02 Oct 2013 14:30:06	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	673,920	865,947	1.2849
01 Oct 2013 13:33:19	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	648,000	831,922	1.2838
30 Sep 2013 12:47:46	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	622,080	798,549	1.2837
27 Sep 2013 11:48:44	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	596,160	766,538	1.2858
26 Sep 2013 11:43:31	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	570,240	733,935	1.2871
25 Sep 2013 10:36:15	1241522	15896387	hadcm3n_3l1k_1980_40_008281360_3	544,320	699,531	1.2851