Task 13346485

Name	hadcm3n_yifh_1940_40_007413747_2
Workunit	7611377
Created	8 Sep 2011, 20:13:41 UTC
Sent	8 Sep 2011, 20:13:44 UTC
Report deadline	9 Dec 2011, 3:40:55 UTC
Received	24 Sep 2011, 23:06:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	840795
Run time	4 days 10 hours 47 min 2 sec
CPU time	4 days 6 hours 2 min 40 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> N Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:49:21 (2608): No heartbeat from core client for 30 sec - exiting 01:49:22 (2608): No heartbeat from core client for 30 sec - exiting 01:49:23 (2608): No heartbeat from core client for 30 sec - exiting 01:49:24 (2608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6664, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6664, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76BEFC56 Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 09/18/11 11:57:57 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\ProgramData\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_yifh_1940_40_007413747;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77a90000 00128000 C:\Windows\system32\ntdll.dll (6.0.6002.18327) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 76bb0000 000dc000 C:\Windows\system32\kernel32.dll (6.0.6002.18449) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 76440000 0009d000 C:\Windows\system32\USER32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 76230000 0004b000 C:\Windows\system32\GDI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 762b0000 000c6000 C:\Windows\system32\ADVAPI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6002.18005 ModLoad: 769e0000 000c3000 C:\Windows\system32\RPCRT4.dll (6.0.6002.18024) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 6eaf0000 000cb000 C:\Windows\system32\OPENGL32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 76630000 000aa000 C:\Windows\system32\msvcrt.dll (7.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.6002.18005 ModLoad: 73150000 00023000 C:\Windows\system32\GLU32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 67e00000 000e5000 C:\Windows\system32\DDRAW.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 67ab0000 00006000 C:\Windows\system32\DCIMAN32.dll (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18051 ModLoad: 76740000 0018a000 C:\Windows\system32\SETUPAPI.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 Get Product Name Failed. ModLoad: 77c50000 0008d000 C:\Windows\system32\OLEAUT32.dll (6.0.6002.18357) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.0.6002.18357 Company Name : Microsoft Corporation Product Name : Product Version : 6.0.6002.18357 ModLoad: 764e0000 00145000 C:\Windows\system32\ole32.dll (6.0.6002.18277) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 73c70000 0000c000 C:\Windows\system32\dwmapi.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 76280000 00029000 C:\Windows\system32\imagehlp.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 77c30000 0001e000 C:\Windows\system32\IMM32.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 76160000 000c8000 C:\Windows\system32\MSCTF.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 77ce0000 00009000 C:\Windows\system32\LPK.DLL (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18051 ModLoad: 76b30000 0007d000 C:\Windows\system32\USP10.dll (1.626.6002.18244) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.6002.18244 (vistasp2_gdr.100416-0345) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.6002.18244 ModLoad: 00130000 00048000 C:\Windows\system32\guard32.dll (5.5.64714.1382) (-exported- Symbols Loaded) Linked PDB Filename : D:\SVNStorage\Projects\CIS_55_METE\Release\x32\Symbols\guard32.pdb File Version : 5, 5, 195786, 1382 Company Name : COMODO Product Name : COMODO Internet Security Product Version : 5, 5, 195786, 1382 ModLoad: 75fe0000 00008000 C:\Windows\system32\VERSION.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 75fd0000 00007000 C:\Windows\system32\fltlib.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : fltLib.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 62b30000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 6c8a0000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 68bc0000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 695, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 107528, QuotaPeakPagedPoolUsage: 107592 QuotaNonPagedPoolUsage: 3200, QuotaPeakNonPagedPoolUsage: 3200 - Virtual Memory Usage - VirtualSize: 897576960, PeakVirtualSize: 897576960 - Pagefile Usage - PagefileUsage: 34398208, PeakPagefileUsage: 107675648 - Working Set Size - WorkingSetSize: 4808704, PeakWorkingSetSize: 4808704, PageFaultCount: 1194 * Dump of thread ID 7252 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 312002.000000, User Time: 156001.000000, Wait Time: 19820968.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76BEFC56 - Registers - eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=76befc56 esp=1132f9c4 ebp=1132fa14 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 kernel32!RaiseException+0x0 1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 001c2619 1132faec 001c25a1 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 76bfd309 7ffff000 1132ffd4 77ad16c3 7ffff000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77ad16c3 7ffff000 74e98969 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 1132ffd4 77ad1696 007bde7f 7ffff000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7ffff000 00000000 00000030 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 6556 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 19820968.000000 - Registers - eax=0000003c ebx=00000000 ecx=00000069 edx=00000001 esi=337dff48 edi=00000000 eip=77af5ca4 esp=337dff04 ebp=337dff6c cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 337dff6c 76bb1c6c 00000064 00000000 337dff88 00749924 ntdll!KiFastSystemCallRet+0x0 337dff7c 00749924 00000064 337dff94 76bfd309 00000000 kernel32!Sleep+0x0 337dff88 76bfd309 00000000 337dffd4 77ad16c3 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 337dff94 77ad16c3 00000000 56a68969 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 337dffd4 77ad1696 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 337dffec 00000000 00749910 00000000 00000000 33cf0000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6664, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76BEFC56 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 09/18/11 11:58:33 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\ProgramData\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_yifh_1940_40_007413747;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77a90000 00128000 C:\Windows\system32\ntdll.dll (6.0.6002.18327) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 6.0.6002.18327 (vistasp2_gdr.101014-0432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18327 ModLoad: 76bb0000 000dc000 C:\Windows\system32\kernel32.dll (6.0.6002.18449) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 6.0.6002.18449 (vistasp2_gdr.110412-0338) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18449 ModLoad: 76440000 0009d000 C:\Windows\system32\USER32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 76230000 0004b000 C:\Windows\system32\GDI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 762b0000 000c6000 C:\Windows\system32\ADVAPI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 769e0000 000c3000 C:\Windows\system32\RPCRT4.dll (6.0.6002.18024) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 6eaf0000 000cb000 C:\Windows\system32\OPENGL32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 76630000 000aa000 C:\Windows\system32\msvcrt.dll (7.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.6002.18005 ModLoad: 73150000 00023000 C:\Windows\system32\GLU32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 67e00000 000e5000 C:\Windows\system32\DDRAW.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 67ab0000 00006000 C:\Windows\system32\DCIMAN32.dll (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18051 ModLoad: 76740000 0018a000 C:\Windows\system32\SETUPAPI.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 Get Product Name Failed. ModLoad: 77c50000 0008d000 C:\Windows\system32\OLEAUT32.dll (6.0.6002.18357) (-nosymbols- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.0.6002.18357 Company Name : Microsoft Corporation Product Name : Product Version : 6.0.6002.18357 ModLoad: 764e0000 00145000 C:\Windows\system32\ole32.dll (6.0.6002.18277) (-nosymbols- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 73c70000 0000c000 C:\Windows\system32\dwmapi.dll (6.0.6001.18000) (-nosymbols- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 292, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 107528, QuotaPeakPagedPoolUsage: 107592 QuotaNonPagedPoolUsage: 3208, QuotaPeakNonPagedPoolUsage: 3208 - Virtual Memory Usage - VirtualSize: 897576960, PeakVirtualSize: 897576960 - Pagefile Usage - PagefileUsage: 34406400, PeakPagefileUsage: 107679744 - Working Set Size - WorkingSetSize: 4820992, PeakWorkingSetSize: 4820992, PageFaultCount: 1197 * Dump of thread ID 7068 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156001.000000, User Time: 0.000000, Wait Time: 19823252.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76BEFC56 - Registers - eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=76befc56 esp=1132f9c4 ebp=1132fa14 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 kernel32!RaiseException+0x0 1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 00142619 1132faec 001425a1 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 76bfd309 7ffff000 1132ffd4 77ad16c3 7ffff000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77ad16c3 7ffff000 74e91c2b 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 1132ffd4 77ad1696 007bde7f 7ffff000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7ffff000 00000000 00000030 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 9616 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 19823252.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=3393ff48 edi=00000000 eip=77af5ca4 esp=3393ff04 ebp=3393ff6c cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 3393ff6c 76bb1c6c 00000064 00000000 3393ff88 00749924 ntdll!KiFastSystemCallRet+0x0 3393ff7c 00749924 00000064 3393ff94 76bfd309 00000000 kernel32!Sleep+0x0 3393ff88 76bfd309 00000000 3393ffd4 77ad16c3 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3393ff94 77ad16c3 00000000 56481c2b 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 3393ffd4 77ad1696 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3393ffec 00000000 00749910 00000000 00000000 33e50000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6664, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76BEFC56 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 09/18/11 11:58:45 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\ProgramData\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_yifh_1940_40_007413747;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77a90000 00128000 C:\Windows\system32\ntdll.dll (6.0.6002.18327) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 76bb0000 000dc000 C:\Windows\system32\kernel32.dll (6.0.6002.18449) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 76440000 0009d000 C:\Windows\system32\USER32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 76230000 0004b000 C:\Windows\system32\GDI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 762b0000 000c6000 C:\Windows\system32\ADVAPI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6002.18005 ModLoad: 769e0000 000c3000 C:\Windows\system32\RPCRT4.dll (6.0.6002.18024) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6001.18000 ModLoad: 6eaf0000 000cb000 C:\Windows\system32\OPENGL32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6000.16386 ModLoad: 76630000 000aa000 C:\Windows\system32\msvcrt.dll (7.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.6002.18005 ModLoad: 73150000 00023000 C:\Windows\system32\GLU32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 67e00000 000e5000 C:\Windows\system32\DDRAW.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 67ab0000 00006000 C:\Windows\system32\DCIMAN32.dll (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18051 ModLoad: 76740000 0018a000 C:\Windows\system32\SETUPAPI.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 Get Product Name Failed. ModLoad: 77c50000 0008d000 C:\Windows\system32\OLEAUT32.dll (6.0.6002.18357) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.0.6002.18357 Company Name : Microsoft Corporation Product Name : Product Version : 6.0.6002.18357 ModLoad: 764e0000 00145000 C:\Windows\system32\ole32.dll (6.0.6002.18277) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 73c70000 0000c000 C:\Windows\system32\dwmapi.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 76280000 00029000 C:\Windows\system32\imagehlp.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6001.18000 ModLoad: 77c30000 0001e000 C:\Windows\system32\IMM32.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 76160000 000c8000 C:\Windows\system32\MSCTF.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 77ce0000 00009000 C:\Windows\system32\LPK.DLL (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18051 ModLoad: 76b30000 0007d000 C:\Windows\system32\USP10.dll (1.626.6002.18244) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.6002.18244 (vistasp2_gdr.100416-0345) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.6002.18244 ModLoad: 00360000 00048000 C:\Windows\system32\guard32.dll (5.5.64714.1382) (-exported- Symbols Loaded) Linked PDB Filename : D:\SVNStorage\Projects\CIS_55_METE\Release\x32\Symbols\guard32.pdb File Version : 5, 5, 195786, 1382 Company Name : COMODO Product Name : COMODO Internet Security Product Version : 5, 5, 195786, 1382 ModLoad: 75fe0000 00008000 C:\Windows\system32\VERSION.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.0.6002.18005 ModLoad: 75fd0000 00007000 C:\Windows\system32\fltlib.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : fltLib.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : Operaèní systém Microsoft® Windows® Product Version : 6.0.6000.16386 ModLoad: 62b30000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 6c8a0000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 68bc0000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 290, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 107528, QuotaPeakPagedPoolUsage: 107592 QuotaNonPagedPoolUsage: 3208, QuotaPeakNonPagedPoolUsage: 3208 - Virtual Memory Usage - VirtualSize: 897576960, PeakVirtualSize: 897576960 - Pagefile Usage - PagefileUsage: 34402304, PeakPagefileUsage: 107679744 - Working Set Size - WorkingSetSize: 4792320, PeakWorkingSetSize: 4792320, PageFaultCount: 1190 * Dump of thread ID 8592 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156001.000000, User Time: 156001.000000, Wait Time: 19824060.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76BEFC56 - Registers - eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=76befc56 esp=1132f9c4 ebp=1132fa14 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 kernel32!RaiseException+0x0 1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 00122619 1132faec 001225a1 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 76bfd309 7ffff000 1132ffd4 77ad16c3 7ffff000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77ad16c3 7ffff000 74e94d68 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 1132ffd4 77ad1696 007bde7f 7ffff000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7ffff000 00000000 00000030 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 8036 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 19824060.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=338dff48 edi=00000000 eip=77af5ca4 esp=338dff04 ebp=338dff6c cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 338dff6c 76bb1c6c 00000064 00000000 338dff88 00749924 ntdll!KiFastSystemCallRet+0x0 338dff7c 00749924 00000064 338dff94 76bfd309 00000000 kernel32!Sleep+0x0 338dff88 76bfd309 00000000 338dffd4 77ad16c3 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 338dff94 77ad16c3 00000000 56564d68 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 338dffd4 77ad1696 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 338dffec 00000000 00749910 00000000 00000000 33df0000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6664, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:25:30 (7412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:56:56 (7676): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77096E0F read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... 13:55:01 (11032): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8572, selfPID=8572, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77098801 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77098801 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77098801 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77098801 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Sep 2011 07:37:04	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	259,200	351,954	1.3578
20 Sep 2011 18:40:48	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	233,280	316,626	1.3573
20 Sep 2011 07:42:58	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	207,360	280,421	1.3523
19 Sep 2011 12:55:56	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	181,440	245,306	1.3520
19 Sep 2011 03:09:20	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	155,520	210,670	1.3546
17 Sep 2011 16:45:00	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	129,600	175,612	1.3550
15 Sep 2011 12:57:36	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	103,680	141,078	1.3607
15 Sep 2011 03:16:38	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	77,760	106,778	1.3732
13 Sep 2011 19:40:38	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	51,840	71,128	1.3721
09 Sep 2011 17:49:48	840795	13346485	hadcm3n_yifh_1940_40_007413747_2	25,920	34,787	1.3421