Task 17456458

Name	hadcm3s_4doz_1997_2_009161149_1
Workunit	9291485
Created	20 Nov 2014, 15:45:31 UTC
Sent	22 Nov 2014, 13:37:06 UTC
Report deadline	21 Feb 2015, 21:04:17 UTC
Received	28 Dec 2014, 1:13:59 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1258610
Run time	3 days 14 hours 3 min 36 sec
CPU time	2 days 8 hours 31 min 52 sec
Validate state	Valid
Credit	622.08
Device peak FLOPS	3.63 GFLOPS
Application version	UK Met Office HadCM3 short v7.24 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> odLoad: 0000000000400000 0000000000c36000 D:\Boinc-Data\projects\climateprediction.net\hadcm3s_um_7.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000076fa0000 0000000000167000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000751c0000 0000000000140000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.3.9600.17056) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000075c10000 00000000000d0000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000074ec0000 000000000014c000 C:\WINDOWS\SYSTEM32\USER32.dll (6.3.9600.17344) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075570000 0000000000078000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075010000 0000000000012000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.3.9600.16438) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.3.9600.16438 (winblue_gdr.131018-1626) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16438 ModLoad: 0000000074d10000 0000000000107000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.3.9600.17246) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.3.9600.17246 (winblue_gdr.140801-1518) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17246 ModLoad: 0000000075820000 00000000000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9600.16384 ModLoad: 00000000753e0000 000000000003e000 C:\WINDOWS\SYSTEM32\sechost.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074c60000 00000000000b0000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.3.9600.17216) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 00000000749b0000 000000000001d000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.3.9600.16408) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.3.9600.16408 (winblue_gdr.130920-1803) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16408 ModLoad: 00000000749a0000 0000000000009000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074940000 0000000000052000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.3.9600.17120) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 6.3.9600.17120 (winblue_gdr.140509-1516) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17120 ModLoad: 0000000074bd0000 0000000000025000 C:\WINDOWS\system32\IMM32.DLL (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000750c0000 00000000000f7000 C:\WINDOWS\SYSTEM32\MSCTF.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 000000006f100000 000000000007b000 C:\Program Files (x86)\AVG\AVG2015\avghookx.dll (15.0.0.5577) (-nosymbols- Symbols Loaded) Linked PDB Filename : D:\build\nightly\2014_1109_213429_HotFix2015-02_AvCompile\source\build\avcli_smoke_utils\RU_MT_SLL!Win32_vs110\bin\Release_Unicode_MT_SLL_vs110\Win32\avghookx.pdb File Version : 15.0.0.5577 Company Name : AVG Technologies CZ, s.r.o. Product Name : AVG Internet Security Product Version : 15.0.0.5577 ModLoad: 0000000064e40000 0000000000148000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.3.9600.16520) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.3.9600.16520 (winblue_gdr.140127-0329) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16520 ModLoad: 0000000074870000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 4304 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 - Registers - eax=1143fa18 ebx=7ffde000 ecx=00000003 edx=00000000 esi=0080e158 edi=009e849c eip=75c22f71 esp=1143fa18 ebp=1143fa74 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1143fa74 007b2d27 e06d7363 00000001 00000003 1143faa0 KERNELBASE!RaiseException+0x0 1143faac 007afae1 1143fabc 0083a2cc 0080e158 00832824 hadcm3s_um_7.24_windows_intelx8!+0x0 1143facc 0040642d 01000000 00500000 00407892 12e30000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143fefc 007b1c2a 00400000 00000000 002f321c 0000000a hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff8c 751d919f 7ffde000 1143ffdc 76ff0bbb 7ffde000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff98 76ff0bbb 7ffde000 ca9274ad 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 76ff0b91 ffffffff 76fdc9cf 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1143ffec 00000000 007b1c7d 7ffde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 5760 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=007352d0 ebx=33aafd28 ecx=00000000 edx=00000000 esi=00000000 edi=33aafd28 eip=76fda83c esp=33aafce8 ebp=33aafd4c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 33aafd4c 75c1103a 00000064 00000000 33aaff8c 007352f4 ntdll!ZwDelayExecution+0x0 33aafd5c 007352f4 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 33aaff8c 751d919f 00000000 33aaffdc 76ff0bbb 00000000 hadcm3s_um_7.24_windows_intelx8!+0x0 33aaff98 76ff0bbb 00000000 e87b74ad 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 33aaffdc 76ff0b91 ffffffff 76fdc9cf 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 33aaffec 00000000 007352d0 00000000 00000000 12e2009c ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8452, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.2.4 Dump Timestamp : 12/13/14 11:04:05 Install Directory : C:\Program Files (x86)\BOINC\ Data Directory : D:\Boinc-Data Project Symstore : LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Boinc-Data\projects\climateprediction.net\hadcm3s_4doz_1997_2_009161149;D:\Boinc-Data\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c36000 D:\Boinc-Data\projects\climateprediction.net\hadcm3s_um_7.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000076fa0000 0000000000167000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000751c0000 0000000000140000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.3.9600.17056) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000075c10000 00000000000d0000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000074ec0000 000000000014c000 C:\WINDOWS\SYSTEM32\USER32.dll (6.3.9600.17344) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075570000 0000000000078000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075010000 0000000000012000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.3.9600.16438) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.3.9600.16438 (winblue_gdr.131018-1626) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16438 ModLoad: 0000000074d10000 0000000000107000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.3.9600.17246) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.3.9600.17246 (winblue_gdr.140801-1518) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17246 ModLoad: 0000000075820000 00000000000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9600.16384 ModLoad: 00000000753e0000 000000000003e000 C:\WINDOWS\SYSTEM32\sechost.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074c60000 00000000000b0000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.3.9600.17216) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 00000000749b0000 000000000001d000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.3.9600.16408) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.3.9600.16408 (winblue_gdr.130920-1803) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16408 ModLoad: 00000000749a0000 0000000000009000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074940000 0000000000052000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.3.9600.17120) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 6.3.9600.17120 (winblue_gdr.140509-1516) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17120 ModLoad: 0000000074bd0000 0000000000025000 C:\WINDOWS\system32\IMM32.DLL (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000750c0000 00000000000f7000 C:\WINDOWS\SYSTEM32\MSCTF.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 000000006f100000 000000000007b000 C:\Program Files (x86)\AVG\AVG2015\avghookx.dll (15.0.0.5577) (-nosymbols- Symbols Loaded) Linked PDB Filename : D:\build\nightly\2014_1109_213429_HotFix2015-02_AvCompile\source\build\avcli_smoke_utils\RU_MT_SLL!Win32_vs110\bin\Release_Unicode_MT_SLL_vs110\Win32\avghookx.pdb File Version : 15.0.0.5577 Company Name : AVG Technologies CZ, s.r.o. Product Name : AVG Internet Security Product Version : 15.0.0.5577 ModLoad: 0000000064e40000 0000000000148000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.3.9600.16520) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.3.9600.16520 (winblue_gdr.140127-0329) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16520 ModLoad: 0000000074870000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 7028 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 - Registers - eax=1143fa18 ebx=7ffde000 ecx=00000003 edx=00000000 esi=0080e158 edi=009e849c eip=75c22f71 esp=1143fa18 ebp=1143fa74 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1143fa74 007b2d27 e06d7363 00000001 00000003 1143faa0 KERNELBASE!RaiseException+0x0 1143faac 007afae1 1143fabc 0083a2cc 0080e158 00832824 hadcm3s_um_7.24_windows_intelx8!+0x0 1143facc 0040642d 01000000 00500000 00407892 12da0000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143fefc 007b1c2a 00400000 00000000 0024321c 0000000a hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff8c 751d919f 7ffde000 1143ffdc 76ff0bbb 7ffde000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff98 76ff0bbb 7ffde000 d2f46200 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 76ff0b91 ffffffff 76fdc9f4 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1143ffec 00000000 007b1c7d 7ffde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 7716 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=007352d0 ebx=33a1fd28 ecx=00000000 edx=00000000 esi=00000000 edi=33a1fd28 eip=76fda83c esp=33a1fce8 ebp=33a1fd4c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 33a1fd4c 75c1103a 00000064 00000000 33a1ff8c 007352f4 ntdll!ZwDelayExecution+0x0 33a1fd5c 007352f4 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 33a1ff8c 751d919f 00000000 33a1ffdc 76ff0bbb 00000000 hadcm3s_um_7.24_windows_intelx8!+0x0 33a1ff98 76ff0bbb 00000000 f0166200 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 33a1ffdc 76ff0b91 ffffffff 76fdc9f4 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 33a1ffec 00000000 007352d0 00000000 00000000 12d9009c ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8452, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.2.4 Dump Timestamp : 12/13/14 11:04:09 Install Directory : C:\Program Files (x86)\BOINC\ Data Directory : D:\Boinc-Data Project Symstore : LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Boinc-Data\projects\climateprediction.net\hadcm3s_4doz_1997_2_009161149;D:\Boinc-Data\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c36000 D:\Boinc-Data\projects\climateprediction.net\hadcm3s_um_7.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000076fa0000 0000000000167000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000751c0000 0000000000140000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.3.9600.17056) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000075c10000 00000000000d0000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000074ec0000 000000000014c000 C:\WINDOWS\SYSTEM32\USER32.dll (6.3.9600.17344) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075570000 0000000000078000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075010000 0000000000012000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.3.9600.16438) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.3.9600.16438 (winblue_gdr.131018-1626) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16438 ModLoad: 0000000074d10000 0000000000107000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.3.9600.17246) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.3.9600.17246 (winblue_gdr.140801-1518) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17246 ModLoad: 0000000075820000 00000000000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9600.16384 ModLoad: 00000000753e0000 000000000003e000 C:\WINDOWS\SYSTEM32\sechost.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074c60000 00000000000b0000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.3.9600.17216) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 00000000749b0000 000000000001d000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.3.9600.16408) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.3.9600.16408 (winblue_gdr.130920-1803) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16408 ModLoad: 00000000749a0000 0000000000009000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074940000 0000000000052000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.3.9600.17120) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 6.3.9600.17120 (winblue_gdr.140509-1516) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17120 ModLoad: 0000000074bd0000 0000000000025000 C:\WINDOWS\system32\IMM32.DLL (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000750c0000 00000000000f7000 C:\WINDOWS\SYSTEM32\MSCTF.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 000000006f100000 000000000007b000 C:\Program Files (x86)\AVG\AVG2015\avghookx.dll (15.0.0.5577) (-nosymbols- Symbols Loaded) Linked PDB Filename : D:\build\nightly\2014_1109_213429_HotFix2015-02_AvCompile\source\build\avcli_smoke_utils\RU_MT_SLL!Win32_vs110\bin\Release_Unicode_MT_SLL_vs110\Win32\avghookx.pdb File Version : 15.0.0.5577 Company Name : AVG Technologies CZ, s.r.o. Product Name : AVG Internet Security Product Version : 15.0.0.5577 ModLoad: 0000000064e40000 0000000000148000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.3.9600.16520) (-nosymbols- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.3.9600.16520 (winblue_gdr.140127-0329) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16520 ModLoad: 0000000074870000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 9204 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 - Registers - eax=1143fa18 ebx=7ffde000 ecx=00000003 edx=00000000 esi=0080e158 edi=009e849c eip=75c22f71 esp=1143fa18 ebp=1143fa70 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1143fa70 007b2d27 e06d7363 00000001 00000003 1143fa9c KERNELBASE!RaiseException+0x0 1143faa8 007afae1 1143fab8 0083a2cc 0080e158 00832824 hadcm3s_um_7.24_windows_intelx8!+0x0 1143fac8 0040643c 04600000 01000000 00500000 00407892 hadcm3s_um_7.24_windows_intelx8!+0x0 1143fefc 007b1c2a 00400000 00000000 002f321c 0000000a hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff8c 751d919f 7ffde000 1143ffdc 76ff0bbb 7ffde000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff98 76ff0bbb 7ffde000 719d6f3b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 76ff0b91 ffffffff 76fdc9d8 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1143ffec 00000000 007b1c7d 7ffde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 6012 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=007352d0 ebx=3386fd28 ecx=00000000 edx=00000000 esi=00000000 edi=3386fd28 eip=76fda83c esp=3386fce8 ebp=3386fd4c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 3386fd4c 75c1103a 00000064 00000000 3386ff8c 007352f4 ntdll!ZwDelayExecution+0x0 3386fd5c 007352f4 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 3386ff8c 751d919f 00000000 3386ffdc 76ff0bbb 00000000 hadcm3s_um_7.24_windows_intelx8!+0x0 3386ff98 76ff0bbb 00000000 53586f3b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 3386ffdc 76ff0b91 ffffffff 76fdc9d8 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3386ffec 00000000 007352d0 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8452, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.2.4 Dump Timestamp : 12/13/14 11:04:20 Install Directory : C:\Program Files (x86)\BOINC\ Data Directory : D:\Boinc-Data Project Symstore : LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Boinc-Data\projects\climateprediction.net\hadcm3s_4doz_1997_2_009161149;D:\Boinc-Data\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c36000 D:\Boinc-Data\projects\climateprediction.net\hadcm3s_um_7.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000076fa0000 0000000000167000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000751c0000 0000000000140000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.3.9600.17056) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000075c10000 00000000000d0000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000074ec0000 000000000014c000 C:\WINDOWS\SYSTEM32\USER32.dll (6.3.9600.17344) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075570000 0000000000078000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075010000 0000000000012000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.3.9600.16438) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.3.9600.16438 (winblue_gdr.131018-1626) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16438 ModLoad: 0000000074d10000 0000000000107000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.3.9600.17246) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.3.9600.17246 (winblue_gdr.140801-1518) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17246 ModLoad: 0000000075820000 00000000000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9600.16384 ModLoad: 00000000753e0000 000000000003e000 C:\WINDOWS\SYSTEM32\sechost.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074c60000 00000000000b0000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.3.9600.17216) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 00000000749b0000 000000000001d000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.3.9600.16408) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.3.9600.16408 (winblue_gdr.130920-1803) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16408 ModLoad: 00000000749a0000 0000000000009000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074940000 0000000000052000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.3.9600.17120) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 6.3.9600.17120 (winblue_gdr.140509-1516) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17120 ModLoad: 0000000074bd0000 0000000000025000 C:\WINDOWS\system32\IMM32.DLL (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000750c0000 00000000000f7000 C:\WINDOWS\SYSTEM32\MSCTF.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 000000006f100000 000000000007b000 C:\Program Files (x86)\AVG\AVG2015\avghookx.dll (15.0.0.5577) (-nosymbols- Symbols Loaded) Linked PDB Filename : D:\build\nightly\2014_1109_213429_HotFix2015-02_AvCompile\source\build\avcli_smoke_utils\RU_MT_SLL!Win32_vs110\bin\Release_Unicode_MT_SLL_vs110\Win32\avghookx.pdb File Version : 15.0.0.5577 Company Name : AVG Technologies CZ, s.r.o. Product Name : AVG Internet Security Product Version : 15.0.0.5577 ModLoad: 0000000064e40000 0000000000148000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.3.9600.16520) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.3.9600.16520 (winblue_gdr.140127-0329) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16520 ModLoad: 0000000074870000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 4896 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 - Registers - eax=1143fa18 ebx=7ffde000 ecx=00000003 edx=00000000 esi=0080e158 edi=009e849c eip=75c22f71 esp=1143fa18 ebp=1143fa74 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1143fa74 007b2d27 e06d7363 00000001 00000003 1143faa0 KERNELBASE!RaiseException+0x0 1143faac 007afae1 1143fabc 0083a2cc 0080e158 00832824 hadcm3s_um_7.24_windows_intelx8!+0x0 1143facc 0040642d 01000000 00500000 00407892 12cc0000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143fefc 007b1c2a 00400000 00000000 115e321c 0000000a hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff8c 751d919f 7ffde000 1143ffdc 76ff0bbb 7ffde000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff98 76ff0bbb 7ffde000 6cc68f3f 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 76ff0b91 ffffffff 76fdc9f5 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1143ffec 00000000 007b1c7d 7ffde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 5460 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=007352d0 ebx=3393fd28 ecx=00000000 edx=00000000 esi=00000000 edi=3393fd28 eip=76fda83c esp=3393fce8 ebp=3393fd4c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 3393fd4c 75c1103a 00000064 00000000 3393ff8c 007352f4 ntdll!ZwDelayExecution+0x0 3393fd5c 007352f4 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 3393ff8c 751d919f 00000000 3393ffdc 76ff0bbb 00000000 hadcm3s_um_7.24_windows_intelx8!+0x0 3393ff98 76ff0bbb 00000000 4e168f3f 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 3393ffdc 76ff0b91 ffffffff 76fdc9f5 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3393ffec 00000000 007352d0 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8452, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.2.4 Dump Timestamp : 12/13/14 11:04:26 Install Directory : C:\Program Files (x86)\BOINC\ Data Directory : D:\Boinc-Data Project Symstore : LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Boinc-Data\projects\climateprediction.net\hadcm3s_4doz_1997_2_009161149;D:\Boinc-Data\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c36000 D:\Boinc-Data\projects\climateprediction.net\hadcm3s_um_7.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000076fa0000 0000000000167000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000751c0000 0000000000140000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.3.9600.17056) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000075c10000 00000000000d0000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000074ec0000 000000000014c000 C:\WINDOWS\SYSTEM32\USER32.dll (6.3.9600.17344) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075570000 0000000000078000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075010000 0000000000012000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.3.9600.16438) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.3.9600.16438 (winblue_gdr.131018-1626) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16438 ModLoad: 0000000074d10000 0000000000107000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.3.9600.17246) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.3.9600.17246 (winblue_gdr.140801-1518) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17246 ModLoad: 0000000075820000 00000000000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9600.16384 ModLoad: 00000000753e0000 000000000003e000 C:\WINDOWS\SYSTEM32\sechost.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074c60000 00000000000b0000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.3.9600.17216) (-nosymbols- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 00000000749b0000 000000000001d000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.3.9600.16408) (-nosymbols- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.3.9600.16408 (winblue_gdr.130920-1803) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16408 ModLoad: 00000000749a0000 0000000000009000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.3.9600.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074940000 0000000000052000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.3.9600.17120) (-nosymbols- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 6.3.9600.17120 (winblue_gdr.140509-1516) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17120 ModLoad: 0000000074bd0000 0000000000025000 C:\WINDOWS\system32\IMM32.DLL (6.3.9600.17031) (-nosymbols- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 7588 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 - Registers - eax=1143fa18 ebx=7ffde000 ecx=00000003 edx=00000000 esi=0080e158 edi=009e849c eip=75c22f71 esp=1143fa18 ebp=1143fa74 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1143fa74 007b2d27 e06d7363 00000001 00000003 1143faa0 KERNELBASE!RaiseException+0x0 1143faac 007afae1 1143fabc 0083a2cc 0080e158 00832824 hadcm3s_um_7.24_windows_intelx8!+0x0 1143facc 0040642d 01000000 00500000 00407892 12cd0000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143fefc 007b1c2a 00400000 00000000 1160321c 0000000a hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff8c 751d919f 7ffde000 1143ffdc 76ff0bbb 7ffde000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff98 76ff0bbb 7ffde000 edf20c38 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 76ff0b91 ffffffff 76fdc9f8 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1143ffec 00000000 007b1c7d 7ffde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 8184 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=007352d0 ebx=3394fd28 ecx=00000000 edx=00000000 esi=00000000 edi=3394fd28 eip=76fda83c esp=3394fce8 ebp=3394fd4c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 3394fd4c 75c1103a 00000064 00000000 3394ff8c 007352f4 ntdll!ZwDelayExecution+0x0 3394fd5c 007352f4 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 3394ff8c 751d919f 00000000 3394ffdc 76ff0bbb 00000000 hadcm3s_um_7.24_windows_intelx8!+0x0 3394ff98 76ff0bbb 00000000 cf250c38 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 3394ffdc 76ff0b91 ffffffff 76fdc9f8 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3394ffec 00000000 007352d0 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8452, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.2.4 Dump Timestamp : 12/13/14 11:04:30 Install Directory : C:\Program Files (x86)\BOINC\ Data Directory : D:\Boinc-Data Project Symstore : LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Boinc-Data\projects\climateprediction.net\hadcm3s_4doz_1997_2_009161149;D:\Boinc-Data\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c36000 D:\Boinc-Data\projects\climateprediction.net\hadcm3s_um_7.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000076fa0000 0000000000167000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.3.9600.17278) (-nosymbols- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 00000000751c0000 0000000000140000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.3.9600.17056) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000075c10000 00000000000d0000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.3.9600.17278) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 0000000074ec0000 000000000014c000 C:\WINDOWS\SYSTEM32\USER32.dll (6.3.9600.17344) (-nosymbols- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075570000 0000000000078000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.3.9600.17031) (-nosymbols- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000075010000 0000000000012000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.3.9600.16438) (-nosymbols- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.3.9600.16438 (winblue_gdr.131018-1626) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16438 ModLoad: 0000000074d10000 0000000000107000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.3.9600.17246) (-nosymbols- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.3.9600.17246 (winblue_gdr.140801-1518) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17246 ModLoad: 0000000075820000 00000000000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.9600.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9600.16384 ModLoad: 00000000753e0000 000000000003e000 C:\WINDOWS\SYSTEM32\sechost.dll (6.3.9600.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 0000000074c60000 00000000000b0000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.3.9600.17216) (-nosymbols- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 00000000749b0000 000000000001d000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.3.9600.16408) (-nosymbols- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.3.9600.16408 (winblue_gdr.130920-1803) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16408 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 7004 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C22F71 - Registers - eax=1143fa18 ebx=7ffde000 ecx=00000003 edx=00000000 esi=0080e158 edi=009e849c eip=75c22f71 esp=1143fa18 ebp=1143fa74 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1143fa74 007b2d27 e06d7363 00000001 00000003 1143faa0 KERNELBASE!+0x0 1143faac 007afae1 1143fabc 0083a2cc 0080e158 00832824 hadcm3s_um_7.24_windows_intelx8!+0x0 1143facc 0040642d 01000000 00500000 00407892 12be0000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143fefc 007b1c2a 00400000 00000000 0010321c 0000000a hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff8c 751d919f 7ffde000 1143ffdc 76ff0bbb 7ffde000 hadcm3s_um_7.24_windows_intelx8!+0x0 1143ff98 76ff0bbb 7ffde000 695883d0 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 76ff0b91 ffffffff 76fdc9d7 00000000 00000000 ntdll!+0x0 1143ffec 00000000 007b1c7d 7ffde000 00000000 00000000 ntdll!+0x0 * Dump of thread ID 2644 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=007352d0 ebx=3389fd28 ecx=00000000 edx=00000000 esi=00000000 edi=3389fd28 eip=76fda83c esp=3389fce8 ebp=3389fd4c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 3389fd4c 75c1103a 00000064 00000000 3389ff8c 007352f4 ntdll!+0x0 3389fd5c 007352f4 00000064 00000000 00000000 00000000 KERNELBASE!+0x0 3389ff8c 751d919f 00000000 3389ffdc 76ff0bbb 00000000 hadcm3s_um_7.24_windows_intelx8!+0x0 3389ff98 76ff0bbb 00000000 4b9283d0 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 3389ffdc 76ff0b91 ffffffff 76fdc9d7 00000000 00000000 ntdll!+0x0 3389ffec 00000000 007352d0 00000000 00000000 00000000 ntdll!+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8452, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model cra

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Dec 2014 23:56:11	1258610	17456458	hadcm3s_4doz_1997_2_009161149_1	51,840	266,369	5.1383
12 Dec 2014 18:46:28	1258610	17456458	hadcm3s_4doz_1997_2_009161149_1	25,920	78,177	3.0161