Task 15439066

Name	hadcm3n_z7ql_1920_40_008245172_0
Workunit	8400296
Created	19 Nov 2012, 16:33:00 UTC
Sent	19 Nov 2012, 16:33:09 UTC
Report deadline	19 Feb 2013, 0:00:20 UTC
Received	12 May 2013, 7:38:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1156294
Run time	29 days 19 hours 27 min 52 sec
CPU time	26 days 9 hours 16 min 42 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.25 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2832, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4060, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2780, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:59:09 (3608): No heartbeat from core client for 30 sec - exiting 07:59:10 (3608): No heartbeat from core client for 30 sec - exiting 07:59:11 (3608): No heartbeat from core client for 30 sec - exiting 07:59:12 (3608): No heartbeat from core client for 30 sec - exiting 07:59:13 (3608): No heartbeat from core client for 30 sec - exiting 07:59:14 (3608): No heartbeat from core client for 30 sec - exiting 07:59:15 (3608): No heartbeat from core client for 30 sec - exiting 07:59:16 (3608): No heartbeat from core client for 30 sec - exiting 07:59:17 (3608): No heartbeat from core client for 30 sec - exiting 07:59:18 (3608): No heartbeat from core client for 30 sec - exiting 07:59:19 (3608): No heartbeat from core client for 30 sec - exiting 07:59:20 (3608): No heartbeat from core client for 30 sec - exiting 07:59:21 (3608): No heartbeat from core client for 30 sec - exiting 07:59:22 (3608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3084, iMonCtr=1 Model crash detected, will try to restart... 09:33:16 (2056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3012, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5456, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3564, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4660, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=168, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3132, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4136, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4136, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4136, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5380, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4832, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 20:59:02 (4568): No heartbeat from core client for 30 sec - exiting 20:59:03 (4568): No heartbeat from core client for 30 sec - exiting 20:59:04 (4568): No heartbeat from core client for 30 sec - exiting 20:59:05 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4900, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4692, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1696, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3276, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 11:01:23 (4260): No heartbeat from core client for 30 sec - exiting 11:01:24 (4260): No heartbeat from core client for 30 sec - exiting 11:01:25 (4260): No heartbeat from core client for 30 sec - exiting 11:01:26 (4260): No heartbeat from core client for 30 sec - exiting 11:01:27 (4260): No heartbeat from core client for 30 sec - exiting 11:01:28 (4260): No heartbeat from core client for 30 sec - exiting 11:01:29 (4260): No heartbeat from core client for 30 sec - exiting 11:01:30 (4260): No heartbeat from core client for 30 sec - exiting 11:01:31 (4260): No heartbeat from core client for 30 sec - exiting 11:01:32 (4260): No heartbeat from core client for 30 sec - exiting 11:01:34 (4260): No heartbeat from core client for 30 sec - exiting 11:01:35 (4260): No heartbeat from core client for 30 sec - exiting 11:01:36 (4260): No heartbeat from core client for 30 sec - exiting 11:01:37 (4260): No heartbeat from core client for 30 sec - exiting 11:01:38 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3732, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3456, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3448, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4128, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2236, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4872, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1020, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5648, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3492, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 16:22:08 (3476): No heartbeat from core client for 30 sec - exiting 16:22:09 (3476): No heartbeat from core client for 30 sec - exiting 16:22:10 (3476): No heartbeat from core client for 30 sec - exiting 16:22:11 (3476): No heartbeat from core client for 30 sec - exiting 16:22:12 (3476): No heartbeat from core client for 30 sec - exiting 16:22:13 (3476): No heartbeat from core client for 30 sec - exiting 16:22:14 (3476): No heartbeat from core client for 30 sec - exiting 16:22:16 (3476): No heartbeat from core client for 30 sec - exiting 16:22:17 (3476): No heartbeat from core client for 30 sec - exiting 16:22:18 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3492, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4784, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2392, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4224, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4232, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x778D3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773671F3 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 May 2013 12:59:42	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	518,400	2,279,635	4.3974
21 Apr 2013 10:17:32	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	492,480	2,170,313	4.4069
10 Apr 2013 06:52:32	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	466,560	2,061,043	4.4175
01 Apr 2013 14:35:38	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	440,640	1,951,681	4.4292
17 Mar 2013 15:15:32	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	414,720	1,840,683	4.4384
05 Mar 2013 12:51:58	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	388,800	1,723,190	4.4321
01 Mar 2013 09:17:36	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	362,880	1,605,837	4.4253
20 Feb 2013 11:58:42	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	336,960	1,491,606	4.4267
06 Feb 2013 14:45:37	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	311,040	1,378,113	4.4307
31 Jan 2013 16:20:52	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	285,120	1,260,061	4.4194
27 Jan 2013 12:56:49	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	259,200	1,136,985	4.3865
20 Jan 2013 15:05:46	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	233,280	1,019,184	4.3689
17 Jan 2013 14:16:02	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	207,360	900,308	4.3418
10 Jan 2013 19:39:53	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	181,440	786,478	4.3346
07 Jan 2013 08:45:58	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	155,520	669,268	4.3034
01 Jan 2013 12:00:13	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	129,600	550,997	4.2515
25 Dec 2012 16:26:08	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	103,680	433,963	4.1856
19 Dec 2012 15:23:06	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	77,760	320,312	4.1192
14 Dec 2012 12:40:34	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	51,840	202,744	3.9110
26 Nov 2012 13:16:43	1156294	15439066	hadcm3n_z7ql_1920_40_008245172_0	25,920	100,517	3.8780