Task 13749560

Name	hadcm3n_u3hp_1980_40_007614389_2
Workunit	7792519
Created	8 Dec 2011, 2:09:20 UTC
Sent	8 Dec 2011, 2:13:42 UTC
Report deadline	8 Mar 2012, 9:40:53 UTC
Received	15 Jan 2012, 23:59:19 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-529697949 (0xE06D7363) Unknown error code
Computer ID	1172319
Run time	13 days 12 hours 50 min 5 sec
CPU time	13 days 1 hours 4 min 52 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.89 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code -529697949 (0xe06d7363) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9640, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9640, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9640, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:01:45 (6016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:02:28 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:56:33 (608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:10:51 (7352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:42:38 (3868): No heartbeat from core client for 30 sec - exiting 08:42:39 (3868): No heartbeat from core client for 30 sec - exiting 08:42:40 (3868): No heartbeat from core client for 30 sec - exiting 08:42:41 (3868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:53:46 (3316): No heartbeat from core client for 30 sec - exiting 08:53:47 (3316): No heartbeat from core client for 30 sec - exiting 08:53:48 (3316): No heartbeat from core client for 30 sec - exiting 08:53:49 (3316): No heartbeat from core client for 30 sec - exiting 08:53:50 (3316): No heartbeat from core client for 30 sec - exiting 08:53:51 (3316): No heartbeat from core client for 30 sec - exiting 08:53:52 (3316): No heartbeat from core client for 30 sec - exiting 08:53:53 (3316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:54 (3316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:40:23 (4216): No heartbeat from core client for 30 sec - exiting 08:40:24 (4216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773B3A93 read attempt to address 0x40A13427 Engaging BOINC Windows Runtime Debugger... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7996, selfPID=7996, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773B3A93 read attempt to address 0x40A13427 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_u3hp_1980_40_007614389/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773B3772 read attempt to address 0x40A13427 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Jan 2012 09:00:31	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	777,600	1,122,293	1.4433
14 Jan 2012 18:55:59	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	751,680	1,084,105	1.4422
14 Jan 2012 07:01:48	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	725,760	1,045,822	1.4410
13 Jan 2012 20:22:19	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	699,840	1,008,513	1.4411
13 Jan 2012 06:57:16	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	673,920	970,152	1.4396
12 Jan 2012 16:26:20	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	648,000	932,233	1.4386
12 Jan 2012 02:41:35	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	622,080	893,849	1.4369
11 Jan 2012 15:56:20	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	596,160	856,643	1.4369
11 Jan 2012 09:24:05	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	570,240	816,089	1.4311
10 Jan 2012 13:14:27	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	544,320	775,642	1.4250
09 Jan 2012 23:01:01	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	518,400	736,262	1.4203
09 Jan 2012 06:17:42	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	492,480	697,537	1.4164
08 Jan 2012 18:02:16	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	466,560	661,427	1.4177
08 Jan 2012 08:14:46	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	440,640	626,929	1.4228
07 Jan 2012 21:52:26	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	414,720	591,825	1.4270
07 Jan 2012 11:12:43	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	388,800	556,785	1.4321
06 Jan 2012 21:09:30	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	362,880	521,583	1.4373
06 Jan 2012 04:47:13	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	336,960	483,627	1.4353
05 Jan 2012 15:16:16	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	311,040	444,810	1.4301
05 Jan 2012 01:57:48	1172319	13749560	hadcm3n_u3hp_1980_40_007614389_2	285,120	405,696	1.4229