Task 15452433

Name	hadcm3n_zcfn_1880_40_008250743_0
Workunit	8405867
Created	22 Nov 2012, 7:39:52 UTC
Sent	22 Nov 2012, 7:40:34 UTC
Report deadline	21 Feb 2013, 15:07:45 UTC
Received	26 Jan 2013, 20:48:44 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1228981
Run time	16 days 6 hours 44 min 9 sec
CPU time	15 days 13 hours 24 min 40 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:37:07 (7148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:08 (7148): No heartbeat from core client for 30 sec - exiting 20:37:09 (7148): No heartbeat from core client for 30 sec - exiting 20:37:10 (7148): No heartbeat from core client for 30 sec - exiting 20:37:11 (7148): No heartbeat from core client for 30 sec - exiting 20:37:12 (7148): No heartbeat from core client for 30 sec - exiting 20:37:13 (7148): No heartbeat from core client for 30 sec - exiting 20:37:14 (7148): No heartbeat from core client for 30 sec - exiting 20:37:15 (7148): No heartbeat from core client for 30 sec - exiting 20:37:16 (7148): No heartbeat from core client for 30 sec - exiting 20:37:17 (7148): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 20:09:37 (4104): No heartbeat from core client for 30 sec - exiting 20:09:38 (4104): No heartbeat from core client for 30 sec - exiting 20:09:39 (4104): No heartbeat from core client for 30 sec - exiting 20:09:40 (4104): No heartbeat from core client for 30 sec - exiting 20:09:41 (4104): No heartbeat from core client for 30 sec - exiting 20:09:42 (4104): No heartbeat from core client for 30 sec - exiting 20:09:43 (4104): No heartbeat from core client for 30 sec - exiting 20:09:44 (4104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:55:56 (6268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:58:47 (6792): No heartbeat from core client for 30 sec - exiting 07:58:49 (6792): No heartbeat from core client for 30 sec - exiting 07:58:50 (6792): No heartbeat from core client for 30 sec - exiting 07:58:51 (6792): No heartbeat from core client for 30 sec - exiting 07:58:52 (6792): No heartbeat from core client for 30 sec - exiting 07:58:53 (6792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:35:00 (6896): No heartbeat from core client for 30 sec - exiting 20:35:01 (6896): No heartbeat from core client for 30 sec - exiting 20:35:02 (6896): No heartbeat from core client for 30 sec - exiting 20:35:03 (6896): No heartbeat from core client for 30 sec - exiting 20:35:04 (6896): No heartbeat from core client for 30 sec - exiting 20:35:05 (6896): No heartbeat from core client for 30 sec - exiting 20:35:06 (6896): No heartbeat from core client for 30 sec - exiting 20:35:07 (6896): No heartbeat from core client for 30 sec - exiting 20:35:08 (6896): No heartbeat from core client for 30 sec - exiting 20:35:09 (6896): No heartbeat from core client for 30 sec - exiting 20:35:10 (6896): No heartbeat from core client for 30 sec - exiting 20:35:11 (6896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:46:16 (7920): No heartbeat from core client for 30 sec - exiting 09:46:17 (7920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:46:18 (7920): No heartbeat from core client for 30 sec - exiting 09:47:18 (7792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:12:29 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:10:16 (6608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:28:27 (8024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:29:05 (6580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zcfnko.pj92c10 Error converting file to netcdf: dataout/zcfnko.pi92c10 Error converting file to netcdf: dataout/zcfnko.pf92c10 Error converting file to netcdf: dataout/zcfnka.ph92c10 Error converting file to netcdf: dataout/zcfnka.pg92c10 Error converting file to netcdf: dataout/zcfnka.pe92c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:58:06 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:30:37 (4360): No heartbeat from core client for 30 sec - exiting 21:30:38 (4360): No heartbeat from core client for 30 sec - exiting 21:30:39 (4360): No heartbeat from core client for 30 sec - exiting 21:30:40 (4360): No heartbeat from core client for 30 sec - exiting 21:30:41 (4360): No heartbeat from core client for 30 sec - exiting 21:30:42 (4360): No heartbeat from core client for 30 sec - exiting 21:30:43 (4360): No heartbeat from core client for 30 sec - exiting 21:30:44 (4360): No heartbeat from core client for 30 sec - exiting 21:30:45 (4360): No heartbeat from core client for 30 sec - exiting 21:30:46 (4360): No heartbeat from core client for 30 sec - exiting 21:30:47 (4360): No heartbeat from core client for 30 sec - exiting 21:30:48 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7332, iMonCtr=1 Model crash detected, will try to restart... 10:10:00 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4992, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:03:51 (6436): No heartbeat from core client for 30 sec - exiting 00:03:52 (6436): No heartbeat from core client for 30 sec - exiting 00:03:53 (6436): No heartbeat from core client for 30 sec - exiting 00:03:54 (6436): No heartbeat from core client for 30 sec - exiting 00:03:55 (6436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77397373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 21:27:37 (7548): No heartbeat from core client for 30 sec - exiting 21:27:39 (7548): No heartbeat from core client for 30 sec - exiting 21:27:40 (7548): No heartbeat from core client for 30 sec - exiting 21:27:41 (7548): No heartbeat from core client for 30 sec - exiting 21:27:42 (7548): No heartbeat from core client for 30 sec - exiting 21:27:43 (7548): No heartbeat from core client for 30 sec - exiting 21:27:44 (7548): No heartbeat from core client for 30 sec - exiting 21:27:45 (7548): No heartbeat from core client for 30 sec - exiting 21:27:46 (7548): No heartbeat from core client for 30 sec - exiting 21:27:47 (7548): No heartbeat from core client for 30 sec - exiting 21:27:48 (7548): No heartbeat from core client for 30 sec - exiting 21:27:49 (7548): No heartbeat from core client for 30 sec - exiting 21:27:50 (7548): No heartbeat from core client for 30 sec - exiting 21:27:51 (7548): No heartbeat from core client for 30 sec - exiting 21:27:52 (7548): No heartbeat from core client for 30 sec - exiting 21:27:53 (7548): No heartbeat from core client for 30 sec - exiting 21:27:54 (7548): No heartbeat from core client for 30 sec - exiting 21:29:29 (7548): No heartbeat from core client for 30 sec - exiting 21:29:30 (7548): No heartbeat from core client for 30 sec - exiting 21:29:31 (7548): No heartbeat from core client for 30 sec - exiting 21:29:32 (7548): No heartbeat from core client for 30 sec - exiting 21:29:33 (7548): No heartbeat from core client for 30 sec - exiting 21:29:34 (7548): No heartbeat from core client for 30 sec - exiting 21:29:35 (7548): No heartbeat from core client for 30 sec - exiting 21:29:36 (7548): No heartbeat from core client for 30 sec - exiting 21:29:37 (7548): No heartbeat from core client for 30 sec - exiting 21:29:38 (7548): No heartbeat from core client for 30 sec - exiting 21:29:39 (7548): No heartbeat from core client for 30 sec - exiting 21:29:40 (7548): No heartbeat from core client for 30 sec - exiting 21:29:41 (7548): No heartbeat from core client for 30 sec - exiting 21:29:42 (7548): No heartbeat from core client for 30 sec - exiting 21:29:43 (7548): No heartbeat from core client for 30 sec - exiting 21:29:44 (7548): No heartbeat from core client for 30 sec - exiting 21:29:45 (7548): No heartbeat from core client for 30 sec - exiting 21:29:46 (7548): No heartbeat from core client for 30 sec - exiting 21:29:47 (7548): No heartbeat from core client for 30 sec - exiting 21:29:48 (7548): No heartbeat from core client for 30 sec - exiting 21:29:49 (7548): No heartbeat from core client for 30 sec - exiting 21:29:50 (7548): No heartbeat from core client for 30 sec - exiting 21:29:51 (7548): No heartbeat from core client for 30 sec - exiting 21:29:52 (7548): No heartbeat from core client for 30 sec - exiting 21:29:53 (7548): No heartbeat from core client for 30 sec - exiting 21:29:54 (7548): No heartbeat from core client for 30 sec - exiting 21:29:55 (7548): No heartbeat from core client for 30 sec - exiting 21:29:56 (7548): No heartbeat from core client for 30 sec - exiting 21:29:57 (7548): No heartbeat from core client for 30 sec - exiting 21:29:58 (7548): No heartbeat from core client for 30 sec - exiting 21:29:59 (7548): No heartbeat from core client for 30 sec - exiting 21:30:00 (7548): No heartbeat from core client for 30 sec - exiting 21:30:01 (7548): No heartbeat from core client for 30 sec - exiting 21:30:02 (7548): No heartbeat from core client for 30 sec - exiting 21:30:03 (7548): No heartbeat from core client for 30 sec - exiting 21:30:04 (7548): No heartbeat from core client for 30 sec - exiting 21:30:05 (7548): No heartbeat from core client for 30 sec - exiting 21:30:06 (7548): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77563AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Jan 2013 19:51:36	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	777,600	1,343,251	1.7274
25 Jan 2013 00:08:36	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	751,680	1,297,934	1.7267
21 Jan 2013 22:43:31	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	725,760	1,253,031	1.7265
20 Jan 2013 12:15:16	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	699,840	1,208,084	1.7262
19 Jan 2013 14:05:24	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	673,920	1,164,007	1.7272
18 Jan 2013 11:36:02	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	648,000	1,119,363	1.7274
16 Jan 2013 21:51:51	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	622,080	1,075,761	1.7293
13 Jan 2013 22:45:01	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	596,160	1,030,826	1.7291
12 Jan 2013 17:31:12	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	570,240	986,086	1.7292
11 Jan 2013 20:00:03	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	544,320	941,694	1.7300
29 Dec 2012 11:00:50	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	518,400	895,967	1.7283
27 Dec 2012 16:11:04	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	492,480	850,713	1.7274
26 Dec 2012 19:38:56	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	466,560	804,823	1.7250
25 Dec 2012 22:16:25	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	440,640	760,608	1.7261
23 Dec 2012 23:08:47	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	414,720	716,598	1.7279
23 Dec 2012 10:38:28	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	388,800	673,057	1.7311
22 Dec 2012 13:15:16	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	362,880	628,236	1.7313
17 Dec 2012 11:15:10	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	336,960	582,656	1.7292
16 Dec 2012 13:25:30	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	311,040	539,200	1.7335
15 Dec 2012 15:50:52	1228981	15452433	hadcm3n_zcfn_1880_40_008250743_0	285,120	493,145	1.7296