Task 13119928

Name	hadcm3n_yjf6_1900_40_007358028_1
Workunit	7555458
Created	6 Jul 2011, 14:56:54 UTC
Sent	8 Jul 2011, 18:26:17 UTC
Report deadline	8 Oct 2011, 1:53:28 UTC
Received	11 Aug 2011, 12:29:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1273691
Run time	8 days 21 hours 15 min 40 sec
CPU time	8 days 12 hours 15 min 23 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.95 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:58:22 (4136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:58:12 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:25:29 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CNo Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4420, selfPID=4420, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/yjf6ko.pjb2c10 Error converting file to netcdf: dataout/yjf6ko.pib2c10 Error converting file to netcdf: dataout/yjf6ko.pfb2c10 Error converting file to netcdf: dataout/yjf6ka.phb2c10 Error converting file to netcdf: dataout/yjf6ka.pgb2c10 Error converting file to netcdf: dataout/yjf6ka.peb2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:06:25 (3960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:24:17 (3100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:25:37 (1772): No heartbeat from core client for 30 sec - exiting 11:25:39 (1772): No heartbeat from core client for 30 sec - exiting 11:25:40 (1772): No heartbeat from core client for 30 sec - exiting 11:25:41 (1772): No heartbeat from core client for 30 sec - exiting 11:25:42 (1772): No heartbeat from core client for 30 sec - exiting 11:25:43 (1772): No heartbeat from core client for 30 sec - exiting 11:25:44 (1772): No heartbeat from core client for 30 sec - exiting 11:25:45 (1772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4524, iMonCtr=1 Model crash detected, will try to restart... 12:59:04 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:25:40 (3480): No heartbeat from core client for 30 sec - exiting 10:25:42 (3480): No heartbeat from core client for 30 sec - exiting 10:25:43 (3480): No heartbeat from core client for 30 sec - exiting 10:25:44 (3480): No heartbeat from core client for 30 sec - exiting 10:25:45 (3480): No heartbeat from core client for 30 sec - exiting 10:25:46 (3480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:07:53 (5128): No heartbeat from core client for 30 sec - exiting 01:07:54 (5128): No heartbeat from core client for 30 sec - exiting 01:07:55 (5128): No heartbeat from core client for 30 sec - exiting 01:07:56 (5128): No heartbeat from core client for 30 sec - exiting 01:07:57 (5128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:07:58 (5128): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 11:45:24 (1452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BD7353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4272, selfPID=4272, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BD7353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77BD7353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77DE3A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Aug 2011 19:33:33	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	777,600	731,777	0.9411
10 Aug 2011 11:05:11	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	751,680	707,866	0.9417
09 Aug 2011 17:07:29	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	725,760	683,757	0.9421
08 Aug 2011 20:37:55	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	699,840	659,626	0.9425
08 Aug 2011 13:28:25	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	673,920	635,544	0.9431
06 Aug 2011 11:48:41	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	648,000	611,874	0.9443
05 Aug 2011 17:52:23	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	622,080	587,961	0.9452
04 Aug 2011 22:05:01	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	596,160	563,776	0.9457
04 Aug 2011 15:05:16	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	570,240	539,602	0.9463
03 Aug 2011 21:00:00	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	544,320	515,889	0.9478
03 Aug 2011 14:15:09	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	518,400	491,753	0.9486
02 Aug 2011 11:53:10	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	492,480	467,754	0.9498
29 Jul 2011 19:41:49	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	466,560	442,555	0.9485
29 Jul 2011 12:43:43	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	440,640	418,799	0.9504
28 Jul 2011 19:38:16	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	414,720	394,034	0.9501
28 Jul 2011 12:29:09	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	388,800	370,017	0.9517
27 Jul 2011 20:44:20	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	362,880	344,136	0.9483
27 Jul 2011 13:03:54	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	336,960	317,511	0.9423
26 Jul 2011 15:27:01	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	311,040	291,556	0.9374
25 Jul 2011 19:40:20	1157036	13119928	hadcm3n_yjf6_1900_40_007358028_1	285,120	266,724	0.9355