Task 15745049

Name	hadcm3n_4iev_1940_40_008312438_1
Workunit	8463573
Created	21 Apr 2013, 14:39:12 UTC
Sent	21 Apr 2013, 14:39:23 UTC
Report deadline	21 Jul 2013, 22:06:34 UTC
Received	4 Jun 2013, 17:35:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1223840
Run time	6 days 17 hours 38 min 28 sec
CPU time	6 days 16 hours 27 min 37 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.57 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 18:41:36 (3944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:41:37 (3944): No heartbeat from core client for 30 sec - exiting 18:41:38 (3944): No heartbeat from core client for 30 sec - exiting 18:41:39 (3944): No heartbeat from core client for 30 sec - exiting 18:41:40 (3944): No heartbeat from core client for 30 sec - exiting 18:41:41 (3944): No heartbeat from core client for 30 sec - exiting 18:41:42 (3944): No heartbeat from core client for 30 sec - exiting 18:41:43 (3944): No heartbeat from core client for 30 sec - exiting 18:41:44 (3944): No heartbeat from core client for 30 sec - exiting 18:41:45 (3944): No heartbeat from core client for 30 sec - exiting 18:41:46 (3944): No heartbeat from core client for 30 sec - exiting 18:46:36 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:46:37 (4848): No heartbeat from core client for 30 sec - exiting 18:46:38 (4848): No heartbeat from core client for 30 sec - exiting 18:46:39 (4848): No heartbeat from core client for 30 sec - exiting 18:46:40 (4848): No heartbeat from core client for 30 sec - exiting 18:46:41 (4848): No heartbeat from core client for 30 sec - exiting 18:46:42 (4848): No heartbeat from core client for 30 sec - exiting 18:46:43 (4848): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2804, iMonCtr=1 Model crash detected, will try to restart... 15:49:03 (3012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:57:04 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:21:02 (2352): No heartbeat from core client for 30 sec - exiting 09:21:04 (2352): No heartbeat from core client for 30 sec - exiting 09:21:05 (2352): No heartbeat from core client for 30 sec - exiting 09:21:06 (2352): No heartbeat from core client for 30 sec - exiting 09:21:07 (2352): No heartbeat from core client for 30 sec - exiting 09:21:08 (2352): No heartbeat from core client for 30 sec - exiting 09:21:09 (2352): No heartbeat from core client for 30 sec - exiting 09:21:10 (2352): No heartbeat from core client for 30 sec - exiting 09:21:11 (2352): No heartbeat from core client for 30 sec - exiting 09:21:12 (2352): No heartbeat from core client for 30 sec - exiting 09:21:14 (2352): No heartbeat from core client for 30 sec - exiting 09:21:15 (2352): No heartbeat from core client for 30 sec - exiting 09:21:16 (2352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:21:17 (2352): No heartbeat from core client for 30 sec - exiting 09:21:18 (2352): No heartbeat from core client for 30 sec - exiting 09:21:19 (2352): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:39:11 (2948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:39:12 (2948): No heartbeat from core client for 30 sec - exiting 14:39:13 (2948): No heartbeat from core client for 30 sec - exiting 14:39:14 (2948): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... 17:00:55 (2972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:40:01 (2852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:40:02 (2852): No heartbeat from core client for 30 sec - exiting 15:40:03 (2852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:50:27 (3236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:20:47 (2880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:20:48 (2880): No heartbeat from core client for 30 sec - exiting 16:20:49 (2880): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:35:36 (3584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3840, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:52:41 (3060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:22:50 (3056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:22:51 (3056): No heartbeat from core client for 30 sec - exiting 13:22:52 (3056): No heartbeat from core client for 30 sec - exiting 13:22:53 (3056): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2720, iMonCtr=1 Model crash detected, will try to restart... 15:38:28 (2968): No heartbeat from core client for 30 sec - exiting 15:38:29 (2968): No heartbeat from core client for 30 sec - exiting 15:38:30 (2968): No heartbeat from core client for 30 sec - exiting 15:38:31 (2968): No heartbeat from core client for 30 sec - exiting 15:38:32 (2968): No heartbeat from core client for 30 sec - exiting 15:38:33 (2968): No heartbeat from core client for 30 sec - exiting 15:38:34 (2968): No heartbeat from core client for 30 sec - exiting 15:38:35 (2968): No heartbeat from core client for 30 sec - exiting 15:38:36 (2968): No heartbeat from core client for 30 sec - exiting 15:38:37 (2968): No heartbeat from core client for 30 sec - exiting 15:38:38 (2968): No heartbeat from core client for 30 sec - exiting 15:38:39 (2968): No heartbeat from core client for 30 sec - exiting 15:38:40 (2968): No heartbeat from core client for 30 sec - exiting 15:38:41 (2968): No heartbeat from core client for 30 sec - exiting 15:38:42 (2968): No heartbeat from core client for 30 sec - exiting 15:38:43 (2968): No heartbeat from core client for 30 sec - exiting 15:38:44 (2968): No heartbeat from core client for 30 sec - exiting 15:38:45 (2968): No heartbeat from core client for 30 sec - exiting 15:38:46 (2968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C33AB3 read attempt to address 0x40A451DE Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77573AB3 read attempt to address 0x40A451DE Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Jun 2013 12:16:24	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	777,600	573,551	0.7376
03 Jun 2013 11:46:00	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	751,680	554,692	0.7379
01 Jun 2013 06:06:57	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	725,760	535,300	0.7376
30 May 2013 06:13:09	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	699,840	516,173	0.7376
28 May 2013 10:49:26	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	673,920	496,967	0.7374
26 May 2013 08:55:05	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	648,000	477,809	0.7374
24 May 2013 11:43:50	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	622,080	458,729	0.7374
23 May 2013 11:13:55	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	596,160	439,606	0.7374
22 May 2013 06:42:32	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	570,240	420,473	0.7374
21 May 2013 07:27:30	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	544,320	401,351	0.7373
20 May 2013 11:21:47	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	518,400	382,243	0.7374
18 May 2013 16:06:45	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	492,480	362,930	0.7369
17 May 2013 15:27:58	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	466,560	343,843	0.7370
16 May 2013 08:52:21	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	440,640	324,647	0.7368
14 May 2013 15:00:55	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	414,720	305,504	0.7367
13 May 2013 07:17:58	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	388,800	286,385	0.7366
11 May 2013 13:04:45	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	362,880	267,244	0.7365
09 May 2013 15:10:47	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	336,960	248,093	0.7363
07 May 2013 15:15:37	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	311,040	228,955	0.7361
06 May 2013 15:00:33	1223840	15745049	hadcm3n_4iev_1940_40_008312438_1	285,120	209,808	0.7359