Task 16183645

Name	hadcm3n_o4ok_1980_40_008385339_4
Workunit	8536198
Created	31 Dec 2013, 18:06:32 UTC
Sent	31 Dec 2013, 18:06:35 UTC
Report deadline	2 Apr 2014, 1:33:46 UTC
Received	13 Mar 2014, 10:41:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1208393
Run time	17 days 15 hours 59 min 1 sec
CPU time	14 days 23 hours 29 min 32 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.15 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:43:28 (7740): No heartbeat from core client for 30 sec - exiting 09:43:30 (7740): No heartbeat from core client for 30 sec - exiting 09:43:31 (7740): No heartbeat from core client for 30 sec - exiting 09:43:32 (7740): No heartbeat from core client for 30 sec - exiting 09:43:33 (7740): No heartbeat from core client for 30 sec - exiting 09:43:34 (7740): No heartbeat from core client for 30 sec - exiting 09:43:35 (7740): No heartbeat from core client for 30 sec - exiting 09:43:36 (7740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:06:37 (3060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:44:28 (9588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:39:46 (15828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:06:54 (15804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:55 (15804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:35:38 (1420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:11:45 (12168): No heartbeat from core client for 30 sec - exiting 17:11:46 (12168): No heartbeat from core client for 30 sec - exiting 17:11:48 (12168): No heartbeat from core client for 30 sec - exiting 17:11:49 (12168): No heartbeat from core client for 30 sec - exiting 17:11:50 (12168): No heartbeat from core client for 30 sec - exiting 17:11:51 (12168): No heartbeat from core client for 30 sec - exiting 17:11:52 (12168): No heartbeat from core client for 30 sec - exiting 17:11:53 (12168): No heartbeat from core client for 30 sec - exiting 17:11:54 (12168): No heartbeat from core client for 30 sec - exiting 17:11:55 (12168): No heartbeat from core client for 30 sec - exiting 17:11:56 (12168): No heartbeat from core client for 30 sec - exiting 17:11:57 (12168): No heartbeat from core client for 30 sec - exiting 17:11:58 (12168): No heartbeat from core client for 30 sec - exiting 17:11:59 (12168): No heartbeat from core client for 30 sec - exiting 17:12:00 (12168): No heartbeat from core client for 30 sec - exiting 17:12:01 (12168): No heartbeat from core client for 30 sec - exiting 17:12:02 (12168): No heartbeat from core client for 30 sec - exiting 17:12:03 (12168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:05:23 (10964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:17:07 (11840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4692, iMonCtr=1 Model crash detected, will try to restart... 08:08:54 (6884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:09:47 (6884): No heartbeat from core client for 30 sec - exiting 08:09:48 (6884): No heartbeat from core client for 30 sec - exiting 08:09:49 (6884): No heartbeat from core client for 30 sec - exiting 08:09:50 (6884): No heartbeat from core client for 30 sec - exiting 08:09:51 (6884): No heartbeat from core client for 30 sec - exiting 08:09:52 (6884): No heartbeat from core client for 30 sec - exiting 08:09:53 (6884): No heartbeat from core client for 30 sec - exiting 08:09:54 (6884): No heartbeat from core client for 30 sec - exiting 08:09:55 (6884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:55:55 (2220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:56:08 (2220): No heartbeat from core client for 30 sec - exiting 09:56:09 (2220): No heartbeat from core client for 30 sec - exiting 09:56:10 (2220): No heartbeat from core client for 30 sec - exiting 09:56:11 (2220): No heartbeat from core client for 30 sec - exiting 09:56:12 (2220): No heartbeat from core client for 30 sec - exiting 09:56:14 (2220): No heartbeat from core client for 30 sec - exiting 09:56:15 (2220): No heartbeat from core client for 30 sec - exiting 09:56:16 (2220): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4520, iMonCtr=1 Model crash detected, will try to restart... 16:00:02 (4692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:31 (4692): No heartbeat from core client for 30 sec - exiting 16:00:32 (4692): No heartbeat from core client for 30 sec - exiting 16:00:33 (4692): No heartbeat from core client for 30 sec - exiting 16:00:34 (4692): No heartbeat from core client for 30 sec - exiting 16:00:35 (4692): No heartbeat from core client for 30 sec - exiting 16:00:36 (4692): No heartbeat from core client for 30 sec - exiting 16:00:37 (4692): No heartbeat from core client for 30 sec - exiting 16:00:38 (4692): No heartbeat from core client for 30 sec - exiting 16:00:39 (4692): No heartbeat from core client for 30 sec - exiting 16:00:40 (4692): No heartbeat from core client for 30 sec - exiting 16:00:41 (4692): No heartbeat from core client for 30 sec - exiting 16:00:42 (4692): No heartbeat from core client for 30 sec - exiting 16:00:43 (4692): No heartbeat from core client for 30 sec - exiting 16:00:44 (4692): No heartbeat from core client for 30 sec - exiting 16:00:45 (4692): No heartbeat from core client for 30 sec - exiting 16:00:46 (4692): No heartbeat from core client for 30 sec - exiting 16:00:47 (4692): No heartbeat from core client for 30 sec - exiting 16:00:48 (4692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7796, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4872, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x778E3AC3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x778E7203 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Mar 2014 02:21:44	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	518,400	1,280,550	2.4702
09 Mar 2014 17:43:33	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	492,480	1,195,954	2.4284
06 Mar 2014 20:14:11	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	466,560	1,121,051	2.4028
04 Mar 2014 23:20:31	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	440,640	1,048,558	2.3796
28 Feb 2014 16:20:33	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	414,720	980,413	2.3640
26 Feb 2014 13:47:17	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	388,800	907,461	2.3340
24 Feb 2014 14:57:25	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	362,880	837,878	2.3090
20 Feb 2014 01:55:41	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	336,960	766,523	2.2748
14 Feb 2014 17:03:19	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	311,040	695,444	2.2359
04 Feb 2014 01:27:43	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	285,120	591,679	2.0752
30 Jan 2014 17:07:43	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	259,200	492,086	1.8985
26 Jan 2014 19:55:04	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	233,280	407,692	1.7477
24 Jan 2014 11:36:41	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	207,360	327,076	1.5773
20 Jan 2014 16:25:08	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	181,440	281,370	1.5508
19 Jan 2014 22:43:11	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	155,520	241,797	1.5548
16 Jan 2014 01:51:52	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	129,600	201,125	1.5519
15 Jan 2014 15:42:04	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	103,680	160,008	1.5433
11 Jan 2014 00:29:17	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	77,760	118,046	1.5181
08 Jan 2014 18:57:50	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	51,840	79,440	1.5324
02 Jan 2014 14:32:17	1208393	16183645	hadcm3n_o4ok_1980_40_008385339_4	25,920	39,923	1.5402