Task 15278996

Name	hadcm3n_zd57_1880_40_008200260_0
Workunit	8355384
Created	13 Sep 2012, 5:51:30 UTC
Sent	14 Sep 2012, 5:35:02 UTC
Report deadline	14 Dec 2012, 13:02:13 UTC
Received	19 Nov 2012, 6:01:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1223423
Run time	10 days 3 hours 25 min 39 sec
CPU time	9 days 18 hours 58 min 51 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.99 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3128, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... 23:21:44 (3780): No heartbeat from core client for 30 sec - exiting 23:21:46 (3780): No heartbeat from core client for 30 sec - exiting 23:21:47 (3780): No heartbeat from core client for 30 sec - exiting 23:21:48 (3780): No heartbeat from core client for 30 sec - exiting 23:21:49 (3780): No heartbeat from core client for 30 sec - exiting 23:21:50 (3780): No heartbeat from core client for 30 sec - exiting 23:21:51 (3780): No heartbeat from core client for 30 sec - exiting 23:21:52 (3780): No heartbeat from core client for 30 sec - exiting 23:21:53 (3780): No heartbeat from core client for 30 sec - exiting 23:21:54 (3780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4700, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2792, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4304, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3648, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4016, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3944, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3880, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:59:43 (4724): Can't set up shared mem: -1. Will run in standalone mode. Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zd57ko.pj92c10 Error converting file to netcdf: dataout/zd57ko.pi92c10 Error converting file to netcdf: dataout/zd57ko.pf92c10 Error converting file to netcdf: dataout/zd57ka.ph92c10 Error converting file to netcdf: dataout/zd57ka.pg92c10 Error converting file to netcdf: dataout/zd57ka.pe92c10 Error converting file to netcdf: dataout/zd57ka.pd92c10 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:31:42 (1396): Can't set up shared mem: -1. Will run in standalone mode. 02:31:42 (4632): Can't set up shared mem: -1. Will run in standalone mode. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:18:22 (3808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7712F5C7 read attempt to address 0x409F9496 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77133541 read attempt to address 0x409F94A2 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Nov 2012 19:08:30	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	518,400	833,302	1.6074
16 Nov 2012 05:35:23	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	492,480	792,451	1.6091
13 Nov 2012 04:32:55	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	466,560	753,068	1.6141
11 Nov 2012 06:27:25	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	440,640	713,565	1.6194
07 Nov 2012 07:42:31	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	414,720	672,366	1.6213
05 Nov 2012 03:32:25	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	388,800	630,367	1.6213
01 Nov 2012 06:01:07	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	362,880	587,155	1.6180
28 Oct 2012 05:48:36	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	336,960	544,457	1.6158
24 Oct 2012 06:06:10	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	311,040	502,565	1.6158
19 Oct 2012 06:51:14	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	285,120	460,529	1.6152
17 Oct 2012 05:49:23	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	259,200	418,800	1.6157
14 Oct 2012 05:10:52	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	233,280	377,373	1.6177
09 Oct 2012 18:17:14	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	207,360	336,033	1.6205
06 Oct 2012 06:01:51	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	181,440	293,585	1.6181
02 Oct 2012 04:31:01	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	155,520	251,813	1.6192
28 Sep 2012 06:40:52	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	129,600	211,125	1.6291
25 Sep 2012 04:11:59	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	103,680	171,158	1.6508
23 Sep 2012 04:25:14	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	77,760	131,603	1.6924
20 Sep 2012 06:41:13	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	51,840	88,987	1.7166
17 Sep 2012 14:36:34	1223423	15278996	hadcm3n_zd57_1880_40_008200260_0	25,920	44,106	1.7016