Task 15443629

Name	hadcm3n_zm0b_1880_40_008246982_0
Workunit	8402106
Created	21 Nov 2012, 4:44:38 UTC
Sent	21 Nov 2012, 4:44:40 UTC
Report deadline	20 Feb 2013, 12:11:51 UTC
Received	28 Jan 2013, 3:39:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1146914
Run time	15 days 15 hours 40 min 2 sec
CPU time	15 days 6 hours 12 min 10 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.41 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 10:27:54 (4196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4192, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4716, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5324, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zm0bko.pj88c10 Error converting file to netcdf: dataout/zm0bko.pi88c10 Error converting file to netcdf: dataout/zm0bko.pf88c10 Error converting file to netcdf: dataout/zm0bka.ph88c10 Error converting file to netcdf: dataout/zm0bka.pg88c10 Error converting file to netcdf: dataout/zm0bka.pe88c10 Error converting file to netcdf: dataout/zm0bka.pd88c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4752, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4776, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4880, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3960, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5460, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5748, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4816, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4072, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4336, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4956, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5416, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5416, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zm0bko.pja1c10 Error converting file to netcdf: dataout/zm0bko.pia1c10 Error converting file to netcdf: dataout/zm0bko.pfa1c10 Error converting file to netcdf: dataout/zm0bka.pha1c10 Error converting file to netcdf: dataout/zm0bka.pga1c10 Error converting file to netcdf: dataout/zm0bka.pea1c10 Error converting file to netcdf: dataout/zm0bka.pda1c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5172, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5244, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4352, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=576, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5132, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4012, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4428, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4816, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7721432D read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Jan 2013 01:53:23	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	777,600	1,318,301	1.6953
25 Jan 2013 23:33:19	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	751,680	1,277,084	1.6990
25 Jan 2013 11:55:52	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	725,760	1,235,969	1.7030
24 Jan 2013 03:50:14	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	699,840	1,187,813	1.6973
20 Jan 2013 08:09:32	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	673,920	1,144,220	1.6979
17 Jan 2013 06:54:14	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	648,000	1,098,906	1.6958
15 Jan 2013 05:32:49	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	622,080	1,051,032	1.6895
11 Jan 2013 11:49:49	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	596,160	1,003,359	1.6830
09 Jan 2013 08:10:05	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	570,240	959,319	1.6823
04 Jan 2013 03:49:18	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	544,320	917,990	1.6865
03 Jan 2013 07:40:42	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	518,400	876,274	1.6903
02 Jan 2013 02:08:41	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	492,480	837,067	1.6997
30 Dec 2012 06:59:04	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	466,560	794,754	1.7034
28 Dec 2012 00:52:34	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	440,640	752,031	1.7067
26 Dec 2012 03:57:54	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	414,720	710,442	1.7131
24 Dec 2012 06:32:19	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	388,800	667,809	1.7176
22 Dec 2012 02:13:03	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	362,880	625,292	1.7231
20 Dec 2012 04:25:14	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	336,960	581,125	1.7246
16 Dec 2012 17:01:30	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	311,040	537,972	1.7296
14 Dec 2012 13:36:29	1146914	15443629	hadcm3n_zm0b_1880_40_008246982_0	285,120	499,987	1.7536