Task 15442788

Name	hadcm3n_zebf_1880_40_008246613_1
Workunit	8401737
Created	21 Nov 2012, 2:26:13 UTC
Sent	21 Nov 2012, 2:26:22 UTC
Report deadline	20 Feb 2013, 9:53:33 UTC
Received	11 Jan 2013, 12:54:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1371858
Run time	19 days 16 hours 22 min 16 sec
CPU time	17 days 21 hours 28 min 41 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.85 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:10:21 (7964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:11:55 (3572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:10:46 (2424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:44:52 (4712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:35 (812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:14:06 (9008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1796, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 14:43:57 (7336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:09 (7336): No heartbeat from core client for 30 sec - exiting 14:44:10 (7336): No heartbeat from core client for 30 sec - exiting 14:44:11 (7336): No heartbeat from core client for 30 sec - exiting 14:44:12 (7336): No heartbeat from core client for 30 sec - exiting 14:44:13 (7336): No heartbeat from core client for 30 sec - exiting 14:44:14 (7336): No heartbeat from core client for 30 sec - exiting 14:44:16 (7336): No heartbeat from core client for 30 sec - exiting 14:44:17 (7336): No heartbeat from core client for 30 sec - exiting 14:44:18 (7336): No heartbeat from core client for 30 sec - exiting 14:44:19 (7336): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 19:18:59 (5952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:00 (5952): No heartbeat from core client for 30 sec - exiting 19:19:01 (5952): No heartbeat from core client for 30 sec - exiting 19:19:02 (5952): No heartbeat from core client for 30 sec - exiting 19:19:03 (5952): No heartbeat from core client for 30 sec - exiting 19:19:04 (5952): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:02:16 (8364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:31:20 (7796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:59 (7200): No heartbeat from core client for 30 sec - exiting 19:32:00 (7200): No heartbeat from core client for 30 sec - exiting 19:32:01 (7200): No heartbeat from core client for 30 sec - exiting 19:32:02 (7200): No heartbeat from core client for 30 sec - exiting 19:32:03 (7200): No heartbeat from core client for 30 sec - exiting 19:32:04 (7200): No heartbeat from core client for 30 sec - exiting 19:32:05 (7200): No heartbeat from core client for 30 sec - exiting 19:32:06 (7200): No heartbeat from core client for 30 sec - exiting 19:32:08 (7200): No heartbeat from core client for 30 sec - exiting 19:32:09 (7200): No heartbeat from core client for 30 sec - exiting 19:32:10 (7200): No heartbeat from core client for 30 sec - exiting 19:32:11 (7200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:30:04 (7508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:36:02 (5448): No heartbeat from core client for 30 sec - exiting 09:36:36 (5448): No heartbeat from core client for 30 sec - exiting 09:36:37 (5448): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zebfko.pja8c10 Error converting file to netcdf: dataout/zebfko.pia8c10 Error converting file to netcdf: dataout/zebfko.pfa8c10 Error converting file to netcdf: dataout/zebfka.pha8c10 Error converting file to netcdf: dataout/zebfka.pga8c10 Error converting file to netcdf: dataout/zebfka.pea8c10 Error converting file to netcdf: dataout/zebfka.pda8c10 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C33AB3 read attempt to address 0x408BC23A Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Jan 2013 12:55:11	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	1,036,800	1,546,117	1.4912
09 Jan 2013 08:40:11	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	1,010,880	1,506,710	1.4905
08 Jan 2013 09:52:15	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	984,960	1,466,527	1.4889
07 Jan 2013 13:46:09	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	959,040	1,425,143	1.4860
05 Jan 2013 05:05:03	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	933,120	1,385,145	1.4844
04 Jan 2013 16:12:57	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	907,200	1,344,432	1.4820
03 Jan 2013 22:32:43	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	881,280	1,306,305	1.4823
03 Jan 2013 09:31:06	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	855,360	1,267,423	1.4817
31 Dec 2012 14:05:37	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	829,440	1,225,251	1.4772
25 Dec 2012 14:55:50	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	803,520	1,175,400	1.4628
21 Dec 2012 15:57:19	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	777,600	1,130,171	1.4534
20 Dec 2012 21:14:54	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	751,680	1,084,509	1.4428
19 Dec 2012 13:37:50	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	725,760	1,047,042	1.4427
17 Dec 2012 17:11:11	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	699,840	1,008,379	1.4409
15 Dec 2012 18:06:33	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	673,920	969,566	1.4387
15 Dec 2012 04:17:42	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	648,000	931,136	1.4369
14 Dec 2012 07:28:33	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	622,080	892,503	1.4347
13 Dec 2012 19:38:33	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	596,160	854,765	1.4338
13 Dec 2012 19:38:33	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	570,240	821,215	1.4401
13 Dec 2012 19:38:33	1241481	15442788	hadcm3n_zebf_1880_40_008246613_1	544,320	785,325	1.4428