Task 17518366

Name	hadcm3n_sa6u_1940_40_009107772_1
Workunit	9238108
Created	30 Nov 2014, 0:19:40 UTC
Sent	30 Nov 2014, 0:30:53 UTC
Report deadline	1 Mar 2015, 7:58:04 UTC
Received	8 Mar 2015, 23:32:23 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1337804
Run time	12 days 9 hours 32 min 2 sec
CPU time	12 days 6 hours 33 min 23 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/sa6uko.pje1c10 Error converting file to netcdf: dataout/sa6uko.pie1c10 Error converting file to netcdf: dataout/sa6uko.pfe1c10 Error converting file to netcdf: dataout/sa6uka.phe1c10 Error converting file to netcdf: dataout/sa6uka.pge1c10 Error converting file to netcdf: dataout/sa6uka.pee1c10 Error converting file to netcdf: dataout/sa6uka.pde1c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6384, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:45:04 (7100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:59:25 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:59:26 (5840): No heartbeat from core client for 30 sec - exiting 21:59:27 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6448, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_windows_intelx86.dll LoadLibrary error: 126 error: cannot delete old C:/ProgramData/BOINC/projects/climateprediction.net/txf/Palatino-Italic.txf Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6932, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13920, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9696, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 20:20:32 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:20:33 (5668): No heartbeat from core client for 30 sec - exiting 20:20:34 (5668): No heartbeat from core client for 30 sec - exiting 20:20:35 (5668): No heartbeat from core client for 30 sec - exiting 20:20:36 (5668): No heartbeat from core client for 30 sec - exiting 20:20:37 (5668): No heartbeat from core client for 30 sec - exiting 20:20:38 (5668): No heartbeat from core client for 30 sec - exiting 20:20:39 (5668): No heartbeat from core client for 30 sec - exiting 20:20:41 (5668): No heartbeat from core client for 30 sec - exiting 20:20:42 (5668): No heartbeat from core client for 30 sec - exiting 20:20:43 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4920, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15400, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77053AC3 read attempt to address 0x40B138CC Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77253AC3 read attempt to address 0x40B138CC Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Mar 2015 16:27:30	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	777,600	1,036,521	1.3330
10 Mar 2015 16:04:31	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	751,680	1,001,338	1.3321
01 Mar 2015 12:29:08	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	725,760	966,117	1.3312
25 Feb 2015 18:04:14	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	699,840	930,820	1.3300
23 Feb 2015 15:37:26	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	673,920	895,389	1.3286
22 Feb 2015 16:32:24	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	648,000	860,326	1.3277
12 Feb 2015 14:03:30	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	622,080	825,226	1.3266
10 Feb 2015 12:58:03	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	596,160	790,179	1.3254
04 Feb 2015 11:45:54	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	570,240	755,109	1.3242
31 Jan 2015 15:43:07	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	544,320	720,018	1.3228
23 Jan 2015 18:20:08	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	518,400	684,829	1.3210
21 Jan 2015 17:32:05	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	492,480	649,581	1.3190
19 Jan 2015 15:25:00	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	466,560	614,038	1.3161
16 Jan 2015 16:48:52	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	440,640	579,052	1.3141
13 Jan 2015 16:08:34	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	414,720	544,053	1.3119
01 Jan 2015 23:19:53	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	388,800	509,483	1.3104
30 Dec 2014 19:35:45	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	362,880	477,374	1.3155
25 Dec 2014 23:25:23	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	336,960	444,344	1.3187
19 Dec 2014 10:36:03	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	311,040	408,899	1.3146
18 Dec 2014 17:01:03	1337804	17518366	hadcm3n_sa6u_1940_40_009107772_1	285,120	373,582	1.3103