Task 13142244

Name	hadcm3n_ym38_1900_40_007361486_2
Workunit	7558916
Created	16 Jul 2011, 13:57:54 UTC
Sent	16 Jul 2011, 14:06:54 UTC
Report deadline	15 Oct 2011, 21:34:05 UTC
Received	20 Sep 2011, 6:08:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-529697949 (0xE06D7363) Unknown error code
Computer ID	1050651
Run time	61 days 21 hours 38 min 38 sec
CPU time	35 days 10 hours 30 min 47 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	1.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code -529697949 (0xe06d7363) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:03:48 (3812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:49 (3812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812A6B Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 09/19/11 21:32:17 Install Directory : E:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : E:\Program Files\BOINC\\dbghelp.dll Loaded Library : E:\Program Files\BOINC\\symsrv.dll Loaded Library : E:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( E:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_ym38_1900_40_007361486;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net ModLoad: 00400000 00111000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c920000 00096000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.3520) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.3520 (xpsp_sp2_qfe.090206-1239) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.3520 ModLoad: 7c800000 0011d000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.3541) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.3541 (xpsp_sp2_gdr.090321-1320) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.3541 ModLoad: 77d10000 0008f000 C:\WINDOWS\system32\USER32.dll (5.1.2600.3099) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.3099 (xpsp_sp2_gdr.070308-0222) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.3099 ModLoad: 77ef0000 00048000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.3466) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.3466 (xpsp_sp2_gdr.081022-1254) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 5.1.2600.3466 ModLoad: 77da0000 000a9000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.3520) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.3520 (xpsp_sp2_qfe.090206-1239) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.3520 ModLoad: 77e50000 00091000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.3738) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.3738 (xpsp_sp2_qfe.100813-1646) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 5.1.2600.3738 ModLoad: 76300000 0001d000 C:\WINDOWS\system32\IMM32.DLL (5.1.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 5.1.2600.2180 ModLoad: 62c20000 00009000 C:\WINDOWS\system32\LPK.DLL (5.1.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 5.1.2600.2180 ModLoad: 73fa0000 0006b000 C:\WINDOWS\system32\USP10.dll (1.420.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0420.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0420.2600.2180 ModLoad: 77be0000 00058000 C:\WINDOWS\system32\msvcrt.dll (7.0.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 7.0.2600.2180 ModLoad: 76cb0000 00020000 C:\WINDOWS\system32\NTMARTA.DLL (5.1.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : ntmarta.pdb File Version : 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.2180 ModLoad: 76f30000 0002c000 C:\WINDOWS\system32\WLDAP32.dll (5.1.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : wldap32.pdb File Version : 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.2180 ModLoad: 76990000 0013d000 C:\WINDOWS\system32\ole32.dll (5.1.2600.2726) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 5.1.2600.2726 (xpsp_sp2_gdr.050725-1528) Company Name : Microsoft Corporation Product Name : Microsoft(R) Windows(R) Operating System Product Version : 5.1.2600.2726 ModLoad: 71b70000 00013000 C:\WINDOWS\system32\SAMLIB.dll (5.1.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : samlib.pdb File Version : 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 5.1.2600.2180 ModLoad: 76d70000 00022000 C:\WINDOWS\system32\Apphelp.dll (5.1.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : apphelp.pdb File Version : 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 5.1.2600.2180 ModLoad: 10000000 000a4000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_se_6.07_windows_intelx86.dll (-exported- Symbols Loaded) Linked PDB Filename : d:\cpdn\cpdnboinc\hadcm3\test\projects\reqs.comlab.ox.ac.uk_cpdn\hadcm3n_se_6.07_windows_intelx86.pdb ModLoad: 030c0000 00115000 E:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 032e0000 00048000 E:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 03330000 0003b000 E:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 77bd0000 00008000 C:\WINDOWS\system32\version.dll (5.1.2600.2180) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 5.1.2600.2180 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 312471, Write: 0, Other 84405 - I/O Transfers Counters - Read: 0, Write: 30644, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 44604, QuotaPeakPagedPoolUsage: 52580 QuotaNonPagedPoolUsage: 2536, QuotaPeakNonPagedPoolUsage: 3280 - Virtual Memory Usage - VirtualSize: 58650624, PeakVirtualSize: 109694976 - Pagefile Usage - PagefileUsage: 2756608, PeakPagefileUsage: 85352448 - Working Set Size - WorkingSetSize: 3063808, PeakWorkingSetSize: 88305664, PageFaultCount: 127752 * Dump of thread ID 4936 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 94843752.000000, User Time: 527812512.000000, Wait Time: 22222516.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812A6B - Registers - eax=0013f260 ebx=02fc0048 ecx=00000000 edx=1009a8e8 esi=0013f2e8 edi=013afa00 eip=7c812a6b esp=0013f25c ebp=0013f2b0 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 0013f2b0 1005e730 e06d7363 00000001 00000003 0013f2dc kernel32!RaiseException+0x0 0013f2e8 1005bcdc 0013f2f8 10091fe0 100813f4 1008b230 hadcm3n_se_6.07_windows_intelx8!umtonc+0x0 0013f304 1000fa97 04ebe800 00000010 02fb5578 00000000 hadcm3n_se_6.07_windows_intelx8!umtonc+0x0 0013f31c 100036b8 32a449c9 004bb848 004bb710 0013f71c hadcm3n_se_6.07_windows_intelx8!+0x0 00000000 00000000 00000000 00000000 00000000 00000000 hadcm3n_se_6.07_windows_intelx8!+0x0 * Dump of thread ID 2300 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 11250000.000000, User Time: 24375000.000000, Wait Time: 22222512.000000 - Registers - eax=00bffa1c ebx=00000000 ecx=f8e28739 edx=f8000000 esi=00000000 edi=00bfff70 eip=7c92e514 esp=00bfff40 ebp=00bfff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 00bfff98 7c802451 00000064 00000000 00bfffb4 00413da4 ntdll!KiFastSystemCallRet+0x0 00bfffa8 00413da4 00000064 00bfffec 7c80b699 00000000 kernel32!Sleep+0x0 00bfffb4 7c80b699 00000000 735c6c61 625f6d68 00000000 hadcm3n_6.07_windows_intelx86!+0x0 00bfffec 00000000 00413d90 00000000 00000000 63646168 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Sep 2011 10:59:08	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	829,440	2,972,497	3.5837
15 Sep 2011 03:52:57	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	803,520	2,873,556	3.5762
14 Sep 2011 02:35:13	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	777,600	2,783,445	3.5795
13 Sep 2011 01:13:26	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	751,680	2,693,793	3.5837
10 Sep 2011 19:36:04	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	725,760	2,602,015	3.5852
08 Sep 2011 17:17:36	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	699,840	2,509,515	3.5858
07 Sep 2011 15:30:53	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	673,920	2,417,587	3.5874
06 Sep 2011 15:05:18	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	648,000	2,330,922	3.5971
05 Sep 2011 14:44:56	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	622,080	2,244,400	3.6079
04 Sep 2011 04:34:01	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	596,160	2,152,691	3.6109
01 Sep 2011 07:08:24	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	570,240	2,053,115	3.6004
29 Aug 2011 00:49:50	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	544,320	1,962,091	3.6047
27 Aug 2011 12:53:33	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	518,400	1,871,370	3.6099
26 Aug 2011 03:15:46	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	492,480	1,778,261	3.6108
22 Aug 2011 10:17:45	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	466,560	1,680,211	3.6013
15 Aug 2011 18:01:26	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	440,640	1,579,053	3.5835
11 Aug 2011 17:47:20	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	414,720	1,480,167	3.5691
09 Aug 2011 01:05:53	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	388,800	1,373,246	3.5320
07 Aug 2011 21:47:59	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	362,880	1,279,530	3.5260
06 Aug 2011 21:04:46	1050651	13142244	hadcm3n_ym38_1900_40_007361486_2	336,960	1,192,035	3.5376