Task 15492929

Name	hadcm3n_38fh_1940_40_008264062_0
Workunit	8419186
Created	21 Dec 2012, 7:31:48 UTC
Sent	23 Dec 2012, 5:35:47 UTC
Report deadline	24 Mar 2013, 13:02:58 UTC
Received	4 Feb 2013, 0:15:39 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1181321
Run time	14 days 8 hours 31 min 22 sec
CPU time	12 days 10 hours 41 min 26 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.26 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:03:37 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:03:38 (5480): No heartbeat from core client for 30 sec - exiting 02:04:35 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:04:36 (5384): No heartbeat from core client for 30 sec - exiting 03:03:27 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:28 (5700): No heartbeat from core client for 30 sec - exiting 03:03:29 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5592, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/38fhko.pjf3c10 Error converting file to netcdf: dataout/38fhko.pif3c10 Error converting file to netcdf: dataout/38fhko.pff3c10 Error converting file to netcdf: dataout/38fhka.phf3c10 Error converting file to netcdf: dataout/38fhka.pgf3c10 Error converting file to netcdf: dataout/38fhka.pef3c10 Error converting file to netcdf: dataout/38fhka.pdf3c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:51:11 (1880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:13 (1880): No heartbeat from core client for 30 sec - exiting 12:51:14 (1880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77801FCB read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 18:13:18 (4748): No heartbeat from core client for 30 sec - exiting 18:13:19 (4748): No heartbeat from core client for 30 sec - exiting 18:13:20 (4748): No heartbeat from core client for 30 sec - exiting 18:13:21 (4748): No heartbeat from core client for 30 sec - exiting 18:13:22 (4748): No heartbeat from core client for 30 sec - exiting 18:13:23 (4748): No heartbeat from core client for 30 sec - exiting 18:13:24 (4748): No heartbeat from core client for 30 sec - exiting 18:13:25 (4748): No heartbeat from core client for 30 sec - exiting 18:13:26 (4748): No heartbeat from core client for 30 sec - exiting 18:13:27 (4748): No heartbeat from core client for 30 sec - exiting 18:13:28 (4748): No heartbeat from core client for 30 sec - exiting 18:13:29 (4748): No heartbeat from core client for 30 sec - exiting 18:13:30 (4748): No heartbeat from core client for 30 sec - exiting 18:13:31 (4748): No heartbeat from core client for 30 sec - exiting 18:13:32 (4748): No heartbeat from core client for 30 sec - exiting 18:13:33 (4748): No heartbeat from core client for 30 sec - exiting 18:13:34 (4748): No heartbeat from core client for 30 sec - exiting 18:13:35 (4748): No heartbeat from core client for 30 sec - exiting 18:13:36 (4748): No heartbeat from core client for 30 sec - exiting 18:13:37 (4748): No heartbeat from core client for 30 sec - exiting 18:13:38 (4748): No heartbeat from core client for 30 sec - exiting 18:13:39 (4748): No heartbeat from core client for 30 sec - exiting 18:13:40 (4748): No heartbeat from core client for 30 sec - exiting 18:13:41 (4748): No heartbeat from core client for 30 sec - exiting 18:13:42 (4748): No heartbeat from core client for 30 sec - exiting 18:13:43 (4748): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773B2AB1 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... 18:13:44 (4748): No heartbeat from core client for 30 sec - exiting 18:13:45 (4748): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jan 2013 22:46:51	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	518,400	1,075,404	2.0745
29 Jan 2013 22:46:51	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	492,480	1,021,848	2.0749
22 Jan 2013 03:09:28	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	466,560	968,915	2.0767
20 Jan 2013 22:33:52	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	440,640	915,574	2.0778
20 Jan 2013 00:02:55	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	414,720	862,763	2.0804
19 Jan 2013 05:49:34	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	388,800	809,779	2.0828
10 Jan 2013 08:43:38	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	362,880	758,054	2.0890
04 Jan 2013 03:23:59	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	336,960	702,779	2.0856
03 Jan 2013 06:25:25	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	311,040	646,642	2.0790
02 Jan 2013 10:55:38	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	285,120	590,539	2.0712
01 Jan 2013 15:35:57	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	259,200	533,056	2.0565
31 Dec 2012 09:48:39	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	233,280	477,363	2.0463
30 Dec 2012 15:00:52	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	207,360	421,205	2.0313
28 Dec 2012 23:12:59	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	181,440	366,565	2.0203
28 Dec 2012 04:38:36	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	155,520	310,557	1.9969
27 Dec 2012 09:34:33	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	129,600	253,435	1.9555
26 Dec 2012 21:05:33	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	103,680	198,170	1.9114
26 Dec 2012 21:05:33	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	77,760	144,614	1.8597
24 Dec 2012 12:45:16	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	51,840	92,133	1.7773
23 Dec 2012 20:55:34	1181321	15492929	hadcm3n_38fh_1940_40_008264062_0	25,920	45,663	1.7617