Task 21383499

Name	hadcm3s_su158_190012_120_771_011667774_2
Workunit	11667774
Created	15 Nov 2018, 6:05:16 UTC
Sent	15 Nov 2018, 6:05:58 UTC
Report deadline	28 Oct 2019, 11:25:58 UTC
Received	20 Nov 2018, 18:52:44 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID	1435965
Run time	2 days 22 hours 19 min 1 sec
CPU time	2 days 19 hours 26 min 15 sec
Validate state	Invalid
Credit	3,111.26
Device peak FLOPS	3.13 GFLOPS
Application version	UK Met Office HadCM3 short v8.34 windows_intelx86
Peak working set size	179.98 MB
Peak swap size	203.46 MB
Peak disk usage	108.84 MB
Stderr	<core_client_version>7.12.1</core_client_version> <![CDATA[ <message> finish file present too long</message> <stderr_txt> dll!RtlValidSecurityDescriptor+0x0 1143ffec 00000000 007b20dd 0023c000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 25648 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223554320.000000 - Registers - eax=00000000 ebx=11566308 ecx=00000000 edx=00000000 esi=11566148 edi=11564cd8 eip=77e2bf1c esp=21a5fdc8 ebp=21a5ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 21a5ff80 74988484 11564cd8 74988460 9d7a8e67 21a5ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 21a5ff94 77e2305a 11564cd8 9e35653d 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 21a5ffdc 77e2302a ffffffff 77e3ecb5 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 21a5ffec 00000000 77e0c920 11564cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 24608 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223554320.000000 - Registers - eax=00000000 ebx=11566f38 ecx=00000000 edx=00000000 esi=11566d78 edi=11564cd8 eip=77e2bf1c esp=31e5fdc8 ebp=31e5ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 31e5ff80 74988484 11564cd8 74988460 8d3a8e67 31e5ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 31e5ff94 77e2305a 11564cd8 8e75653d 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 31e5ffdc 77e2302a ffffffff 77e3ecb5 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 31e5ffec 00000000 77e0c920 11564cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 23492 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223554320.000000 - Registers - eax=00000000 ebx=11567178 ecx=00000000 edx=00000000 esi=11566fb8 edi=11564cd8 eip=77e2bf1c esp=4225fdc8 ebp=4225ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 4225ff80 74988484 11564cd8 74988460 fefa8e67 4225ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 4225ff94 77e2305a 11564cd8 fdb5653d 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 4225ffdc 77e2302a ffffffff 77e3ecb5 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 4225ffec 00000000 77e0c920 11564cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 28480 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223554336.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=77e2a5ac esp=648bfcdc ebp=648bfd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 648bfd40 748670af 00000064 00000000 648bff80 007355c4 ntdll!NtDelayExecution+0x0 648bfd50 007355c4 00000064 007355a0 007355a0 00000000 KERNELBASE!Sleep+0x0 648bff80 74988484 00000000 74988460 d8548e67 648bffdc hadcm3s_um_8.34_windows_intelx8!+0x0 648bff94 77e2305a 00000000 db1b653d 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 648bffdc 77e2302a ffffffff 77e3ecb5 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 648bffec 00000000 007355a0 00000000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Debug Message Dump * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26536, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 11/20/18 18:08:38 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_su158_190012_120_771_011667774;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c38000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_um_8.34_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077dc0000 0000000000190000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.254) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074970000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074760000 00000000001e4000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 00000000745f0000 000000000007f000 C:\WINDOWS\System32\SYSFER.DLL (14.0.3856.1100) (-nosymbols- Symbols Loaded) Linked PDB Filename : C:\Bld_area\SEP_14.0-RU1-MP1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 14.0.3856.1100 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 14.0.3856.1100 ModLoad: 0000000074d60000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 00000000779d0000 00000000000bf000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.17134.1 ModLoad: 00000000771e0000 0000000000044000 C:\WINDOWS\System32\sechost.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077c10000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.112) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074680000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074670000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077ab0000 0000000000058000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 0000000076e80000 000000000018d000 C:\WINDOWS\System32\USER32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074950000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077010000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.17134.285 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.285 ModLoad: 0000000077800000 0000000000164000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 00000000746e0000 000000000007d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074a70000 000000000011e000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.17134.319 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.319 ModLoad: 0000000074b90000 0000000000019000 C:\WINDOWS\System32\imagehlp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077230000 0000000000026000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 000000006a490000 0000000000182000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000073e00000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 200, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124616, QuotaPeakPagedPoolUsage: 124616 QuotaNonPagedPoolUsage: 9712, QuotaPeakNonPagedPoolUsage: 9712 - Virtual Memory Usage - VirtualSize: 1796775936, PeakVirtualSize: 1796775936 - Pagefile Usage - PagefileUsage: 33533952, PeakPagefileUsage: 105549824 - Working Set Size - WorkingSetSize: 6144000, PeakWorkingSetSize: 6144000, PageFaultCount: 1601 * Dump of thread ID 6680 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 223555568.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 - Registers - eax=1143fa08 ebx=00364000 ecx=00000003 edx=00000000 esi=0080d148 edi=009e879c eip=748717d2 esp=1143fa08 ebp=1143fa64 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1143fa64 007b3187 e06d7363 00000001 00000003 1143fa90 KERNELBASE!RaiseException+0x0 1143fa9c 007aff41 1143faac 00839454 0080d148 008319c4 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fabc 0040666c 04600000 01000000 00500000 00407aa6 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fef0 007b208a 00400000 00000000 116f509b 0000000a hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff80 74988484 00364000 74988460 d29013db 1143ffdc hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff94 77e2305a 00364000 d1dfd7b4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 77e2302a ffffffff 77e3ec8d 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 1143ffec 00000000 007b20dd 00364000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 27716 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223555552.000000 - Registers - eax=00000000 ebx=116f6308 ecx=00000000 edx=00000000 esi=116f6148 edi=116f4cd8 eip=77e2bf1c esp=21befdc8 ebp=21beff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 21beff80 74988484 116f4cd8 74988460 e26d13db 21beffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 21beff94 77e2305a 116f4cd8 e122d7b4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 21beffdc 77e2302a ffffffff 77e3ec8d 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 21beffec 00000000 77e0c920 116f4cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 14232 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223555552.000000 - Registers - eax=00000000 ebx=116f6f38 ecx=00000000 edx=00000000 esi=116f6d78 edi=116f4cd8 eip=77e2bf1c esp=31fefdc8 ebp=31feff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 31feff80 74988484 116f4cd8 74988460 f22d13db 31feffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 31feff94 77e2305a 116f4cd8 f162d7b4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 31feffdc 77e2302a ffffffff 77e3ec8d 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 31feffec 00000000 77e0c920 116f4cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 28140 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223555552.000000 - Registers - eax=00000000 ebx=116fabd0 ecx=00000000 edx=00000000 esi=116faa10 edi=116f4cd8 eip=77e2bf1c esp=423efdc8 ebp=423eff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 423eff80 74988484 116f4cd8 74988460 81ed13db 423effdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 423eff94 77e2305a 116f4cd8 82a2d7b4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 423effdc 77e2302a ffffffff 77e3ec8d 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 423effec 00000000 77e0c920 116f4cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 24608 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223555568.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=77e2a5ac esp=6485fcdc ebp=6485fd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 6485fd40 748670af 00000064 00000000 6485ff80 007355c4 ntdll!NtDelayExecution+0x0 6485fd50 007355c4 00000064 007355a0 007355a0 00000000 KERNELBASE!Sleep+0x0 6485ff80 74988484 00000000 74988460 a75613db 6485ffdc hadcm3s_um_8.34_windows_intelx8!+0x0 6485ff94 77e2305a 00000000 a419d7b4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 6485ffdc 77e2302a ffffffff 77e3ec8d 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 6485ffec 00000000 007355a0 00000000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26536, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 11/20/18 18:09:05 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_su158_190012_120_771_011667774;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c38000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_um_8.34_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077dc0000 0000000000190000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.254) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074970000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074760000 00000000001e4000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 00000000745f0000 000000000007f000 C:\WINDOWS\System32\SYSFER.DLL (14.0.3856.1100) (-nosymbols- Symbols Loaded) Linked PDB Filename : C:\Bld_area\SEP_14.0-RU1-MP1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 14.0.3856.1100 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 14.0.3856.1100 ModLoad: 0000000074d60000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 00000000779d0000 00000000000bf000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.17134.1 ModLoad: 00000000771e0000 0000000000044000 C:\WINDOWS\System32\sechost.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077c10000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.112) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074680000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074670000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077ab0000 0000000000058000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 0000000076e80000 000000000018d000 C:\WINDOWS\System32\USER32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074950000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077010000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.17134.285 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.285 ModLoad: 0000000077800000 0000000000164000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 00000000746e0000 000000000007d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074a70000 000000000011e000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.17134.319 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.319 ModLoad: 0000000074b90000 0000000000019000 C:\WINDOWS\System32\imagehlp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077230000 0000000000026000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 000000006a3c0000 0000000000182000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000073e00000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 200, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124616, QuotaPeakPagedPoolUsage: 124616 QuotaNonPagedPoolUsage: 9712, QuotaPeakNonPagedPoolUsage: 9712 - Virtual Memory Usage - VirtualSize: 1796759552, PeakVirtualSize: 1796759552 - Pagefile Usage - PagefileUsage: 33533952, PeakPagefileUsage: 105549824 - Working Set Size - WorkingSetSize: 6139904, PeakWorkingSetSize: 6139904, PageFaultCount: 1602 * Dump of thread ID 18216 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 468750.000000, User Time: 0.000000, Wait Time: 223557280.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 - Registers - eax=1143fa08 ebx=0021d000 ecx=00000003 edx=00000000 esi=0080d148 edi=009e879c eip=748717d2 esp=1143fa08 ebp=1143fa64 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1143fa64 007b3187 e06d7363 00000001 00000003 1143fa90 KERNELBASE!RaiseException+0x0 1143fa9c 007aff41 1143faac 00839454 0080d148 008319c4 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fabc 0040666c 04600000 01000000 00500000 00407aa6 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fef0 007b208a 00400000 00000000 114e509b 0000000a hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff80 74988484 0021d000 74988460 798362dc 1143ffdc hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff94 77e2305a 0021d000 7accb9e4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 77e2302a ffffffff 77e3ecb0 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 1143ffec 00000000 007b20dd 0021d000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 17804 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223557280.000000 - Registers - eax=00000000 ebx=114e6308 ecx=00000000 edx=00000000 esi=114e6148 edi=114e4cd8 eip=77e2bf1c esp=21aafdc8 ebp=21aaff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 21aaff80 74988484 114e4cd8 74988460 496a62dc 21aaffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 21aaff94 77e2305a 114e4cd8 4a25b9e4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 21aaffdc 77e2302a ffffffff 77e3ecb0 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 21aaffec 00000000 77e0c920 114e4cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 6680 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223557280.000000 - Registers - eax=00000000 ebx=114e6f38 ecx=00000000 edx=00000000 esi=114e6d78 edi=114e4cd8 eip=77e2bf1c esp=31eafdc8 ebp=31eaff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 31eaff80 74988484 114e4cd8 74988460 592a62dc 31eaffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 31eaff94 77e2305a 114e4cd8 5a65b9e4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 31eaffdc 77e2302a ffffffff 77e3ecb0 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 31eaffec 00000000 77e0c920 114e4cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 14912 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223557280.000000 - Registers - eax=00000000 ebx=114eabc8 ecx=00000000 edx=00000000 esi=114eaa08 edi=114e4cd8 eip=77e2bf1c esp=422afdc8 ebp=422aff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 422aff80 74988484 114e4cd8 74988460 2aea62dc 422affdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 422aff94 77e2305a 114e4cd8 29a5b9e4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 422affdc 77e2302a ffffffff 77e3ecb0 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 422affec 00000000 77e0c920 114e4cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 21284 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223557280.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=77e2a5ac esp=6488fcdc ebp=6488fd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 6488fd40 748670af 00000064 00000000 6488ff80 007355c4 ntdll!NtDelayExecution+0x0 6488fd50 007355c4 00000064 007355a0 007355a0 00000000 KERNELBASE!Sleep+0x0 6488ff80 74988484 00000000 74988460 0c4862dc 6488ffdc hadcm3s_um_8.34_windows_intelx8!+0x0 6488ff94 77e2305a 00000000 0f07b9e4 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 6488ffdc 77e2302a ffffffff 77e3ecb0 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 6488ffec 00000000 007355a0 00000000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26536, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 11/20/18 18:09:20 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_su158_190012_120_771_011667774;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c38000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_um_8.34_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077dc0000 0000000000190000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.254) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074970000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074760000 00000000001e4000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 00000000745f0000 000000000007f000 C:\WINDOWS\System32\SYSFER.DLL (14.0.3856.1100) (-nosymbols- Symbols Loaded) Linked PDB Filename : C:\Bld_area\SEP_14.0-RU1-MP1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 14.0.3856.1100 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 14.0.3856.1100 ModLoad: 0000000074d60000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 00000000779d0000 00000000000bf000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.17134.1 ModLoad: 00000000771e0000 0000000000044000 C:\WINDOWS\System32\sechost.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077c10000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.112) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074680000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074670000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077ab0000 0000000000058000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 0000000076e80000 000000000018d000 C:\WINDOWS\System32\USER32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074950000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077010000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.17134.285 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.285 ModLoad: 0000000077800000 0000000000164000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 00000000746e0000 000000000007d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074a70000 000000000011e000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.17134.319 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.319 ModLoad: 0000000074b90000 0000000000019000 C:\WINDOWS\System32\imagehlp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077230000 0000000000026000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000043bb0000 0000000000182000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000073e00000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 200, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124616, QuotaPeakPagedPoolUsage: 124616 QuotaNonPagedPoolUsage: 9712, QuotaPeakNonPagedPoolUsage: 9712 - Virtual Memory Usage - VirtualSize: 1796808704, PeakVirtualSize: 1796808704 - Pagefile Usage - PagefileUsage: 33533952, PeakPagefileUsage: 105549824 - Working Set Size - WorkingSetSize: 6139904, PeakWorkingSetSize: 6139904, PageFaultCount: 1604 * Dump of thread ID 28192 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 312500.000000, User Time: 312500.000000, Wait Time: 223558272.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 - Registers - eax=1143fa08 ebx=003aa000 ecx=00000003 edx=00000000 esi=0080d148 edi=009e879c eip=748717d2 esp=1143fa08 ebp=1143fa64 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child Suspended CPDN Monitor - Suspend request from BOINC... 1143fa64 007b3187 e06d7363 00000001 00000003 1143fa90 KERNELBASE!RaiseException+0x0 1143fa9c 007aff41 1143faac 00839454 0080d148 008319c4 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fabc 0040666c 04600000 01000000 00500000 00407aa6 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fef0 007b208a 00400000 00000000 1171509b 0000000a hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff80 74988484 003aa000 74988460 d27cd5fd 1143ffdc hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff94 77e2305a 003aa000 d1335693 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 77e2302a ffffffff 77e3ec95 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 1143ffec 00000000 007b20dd 003aa000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 27688 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223558240.000000 - Registers - eax=00000000 ebx=11716308 ecx=00000000 edx=00000000 esi=11716148 edi=11714cd8 eip=77e2bf1c esp=21c0fdc8 ebp=21c0ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 21c0ff80 74988484 11714cd8 74988460 e2ffd5fd 21c0ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 21c0ff94 77e2305a 11714cd8 e1b05693 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 21c0ffdc 77e2302a ffffffff 77e3ec95 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 21c0ffec 00000000 77e0c920 11714cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 19820 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223558240.000000 - Registers - eax=00000000 ebx=11716f38 ecx=00000000 edx=00000000 esi=11716d78 edi=11714cd8 eip=77e2bf1c esp=3200fdc8 ebp=3200ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3200ff80 74988484 11714cd8 74988460 f13fd5fd 3200ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 3200ff94 77e2305a 11714cd8 f2705693 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 3200ffdc 77e2302a ffffffff 77e3ec95 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 3200ffec 00000000 77e0c920 11714cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 23348 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223558240.000000 - Registers - eax=00000000 ebx=1171a298 ecx=00000000 edx=00000000 esi=1171a0d8 edi=11714cd8 eip=77e2bf1c esp=4240fdc8 ebp=4240ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 4240ff80 74988484 11714cd8 74988460 817fd5fd 4240ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 4240ff94 77e2305a 11714cd8 82305693 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 4240ffdc 77e2302a ffffffff 77e3ec95 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 4240ffec 00000000 77e0c920 11714cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 7736 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223558256.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=77e2a5ac esp=649efcdc ebp=649efd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 649efd40 748670af 00000064 00000000 649eff80 007355c4 ntdll!NtDelayExecution+0x0 649efd50 007355c4 00000064 007355a0 007355a0 00000000 KERNELBASE!Sleep+0x0 649eff80 74988484 00000000 74988460 a7a1d5fd 649effdc hadcm3s_um_8.34_windows_intelx8!+0x0 649eff94 77e2305a 00000000 a4ee5693 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 649effdc 77e2302a ffffffff 77e3ec95 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 649effec 00000000 007355a0 00000000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26536, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 11/20/18 18:09:49 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Suspended CPDN Monitor - Suspend request from BOINC... Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_su158_190012_120_771_011667774;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000c38000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3s_um_8.34_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077dc0000 0000000000190000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.254) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074970000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 0000000074760000 00000000001e4000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.17134.228 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.228 ModLoad: 00000000745f0000 000000000007f000 C:\WINDOWS\System32\SYSFER.DLL (14.0.3856.1100) (-nosymbols- Symbols Loaded) Linked PDB Filename : C:\Bld_area\SEP_14.0-RU1-MP1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 14.0.3856.1100 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 14.0.3856.1100 ModLoad: 0000000074d60000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 00000000779d0000 00000000000bf000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.17134.1 ModLoad: 00000000771e0000 0000000000044000 C:\WINDOWS\System32\sechost.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077c10000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.112) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074680000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074670000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077ab0000 0000000000058000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 0000000076e80000 000000000018d000 C:\WINDOWS\System32\USER32.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074950000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077010000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.17134.285 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.285 ModLoad: 0000000077800000 0000000000164000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.345) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.17134.345 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.345 ModLoad: 00000000746e0000 000000000007d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000074a70000 000000000011e000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.319) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.17134.319 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.319 ModLoad: 0000000074b90000 0000000000019000 C:\WINDOWS\System32\imagehlp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000077230000 0000000000026000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 000000006a4a0000 0000000000182000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 ModLoad: 0000000073e00000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.17134.1 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.17134.1 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 200, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124616, QuotaPeakPagedPoolUsage: 124616 QuotaNonPagedPoolUsage: 9712, QuotaPeakNonPagedPoolUsage: 9712 - Virtual Memory Usage - VirtualSize: 1796734976, PeakVirtualSize: 1796734976 - Pagefile Usage - PagefileUsage: 33509376, PeakPagefileUsage: 105549824 - Working Set Size - WorkingSetSize: 6135808, PeakWorkingSetSize: 6135808, PageFaultCount: 1601 * Dump of thread ID 3344 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 156250.000000, Wait Time: 223560144.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x748717D2 - Registers - eax=1143fa08 ebx=00384000 ecx=00000003 edx=00000000 esi=0080d148 edi=009e879c eip=748717d2 esp=1143fa08 ebp=1143fa64 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1143fa64 007b3187 e06d7363 00000001 00000003 1143fa90 KERNELBASE!RaiseException+0x0 1143fa9c 007aff41 1143faac 00839454 0080d148 008319c4 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fabc 0040666c 04600000 01000000 00500000 00407aa6 hadcm3s_um_8.34_windows_intelx8!+0x0 1143fef0 007b208a 00400000 00000000 1155509b 0000000a hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff80 74988484 00384000 74988460 d25b0253 1143ffdc hadcm3s_um_8.34_windows_intelx8!+0x0 1143ff94 77e2305a 00384000 d114a931 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1143ffdc 77e2302a ffffffff 77e3ec91 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 1143ffec 00000000 007b20dd 00384000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 20016 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223560128.000000 - Registers - eax=00000000 ebx=11556308 ecx=00000000 edx=00000000 esi=11556148 edi=11554cd8 eip=77e2bf1c esp=21a4fdc8 ebp=21a4ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 21a4ff80 74988484 11554cd8 74988460 e2bc0253 21a4ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 21a4ff94 77e2305a 11554cd8 e1f3a931 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 21a4ffdc 77e2302a ffffffff 77e3ec91 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 21a4ffec 00000000 77e0c920 11554cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 28140 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223560128.000000 - Registers - eax=00000000 ebx=11556f38 ecx=00000000 edx=00000000 esi=11556d78 edi=11554cd8 eip=77e2bf1c esp=31e4fdc8 ebp=31e4ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 31e4ff80 74988484 11554cd8 74988460 f2fc0253 31e4ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 31e4ff94 77e2305a 11554cd8 f1b3a931 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 31e4ffdc 77e2302a ffffffff 77e3ec91 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 31e4ffec 00000000 77e0c920 11554cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 15452 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223560128.000000 - Registers - eax=00000000 ebx=1155a218 ecx=00000000 edx=00000000 esi=1155a058 edi=11554cd8 eip=77e2bf1c esp=4224fdc8 ebp=4224ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 4224ff80 74988484 11554cd8 74988460 813c0253 4224ffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 4224ff94 77e2305a 11554cd8 8273a931 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 4224ffdc 77e2302a ffffffff 77e3ec91 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 4224ffec 00000000 77e0c920 11554cd8 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Dump of thread ID 9164 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 223560144.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=77e2a5ac esp=6495fcdc ebp=6495fd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 6495fd40 748670af 00000064 00000000 6495ff80 007355c4 ntdll!NtDelayExecution+0x0 6495fd50 007355c4 00000064 007355a0 007355a0 00000000 KERNELBASE!Sleep+0x0 6495ff80 74988484 00000000 74988460 a78d0253 6495ffdc hadcm3s_um_8.34_windows_intelx8!+0x0 6495ff94 77e2305a 00000000 a4c2a931 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 6495ffdc 77e2302a ffffffff 77e3ec91 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 6495ffec 00000000 007355a0 00000000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26536, iMonCtr=1 Model crash detected, will try to

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Nov 2018 14:26:14	1435965	21383499	hadcm3s_su158_190012_120_771_011667774_2	259,272	74,199	0.2862