climateprediction.net home page
Task 16024975

Task 16024975

Name hadcm3n_84du_1980_40_008463638_0
Workunit 8614477
Created 19 Sep 2013, 14:32:49 UTC
Sent 21 Sep 2013, 20:31:16 UTC
Report deadline 22 Dec 2013, 3:58:27 UTC
Received 25 Sep 2013, 13:17:38 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1294177
Run time 2 days 17 hours 31 min 23 sec
CPU time 2 days 8 hours 12 min 22 sec
Validate state Invalid
Credit 2,799.36
Device peak FLOPS 4.02 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>7.0.64</core_client_version>
<![CDATA[
<message>
The device does not recognize the command.
 (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:16:02 (26408): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO                                                                                                                                                                                                      tmp/pipe_dummy                                                                  2048    
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=1
Model crash detected, will try to restart...
Signal 22 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=1
Model crash detected, will try to restart...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C8812F

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 09/25/13 07:48:20
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
Loaded Library    : C:\Program Files\BOINC\\dbghelp.dll
Loaded Library    : C:\Program Files\BOINC\\symsrv.dll
Loaded Library    : C:\Program Files\BOINC\\srcsrv.dll
LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_84du_1980_40_008463638;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 77b60000 0013c000 C:\Windows\SYSTEM32\ntdll.dll (6.1.7601.18205) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 772a0000 000d4000 C:\Windows\system32\kernel32.dll (6.1.7601.18229) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18015

ModLoad: 75c80000 0004b000 C:\Windows\system32\KERNELBASE.dll (6.1.7601.18229) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18015

ModLoad: 76230000 000c9000 C:\Windows\system32\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 77d40000 0004e000 C:\Windows\system32\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 77ca0000 0000a000 C:\Windows\system32\LPK.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : lpk.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 76190000 0009d000 C:\Windows\system32\USP10.dll (1.626.7601.18009) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.18009 (win7sp1_gdr.121121-1431)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.18009

ModLoad: 77630000 000ac000 C:\Windows\system32\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 76330000 000a0000 C:\Windows\system32\ADVAPI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 763d0000 00019000 C:\Windows\SYSTEM32\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 773e0000 000a2000 C:\Windows\system32\RPCRT4.dll (6.1.7601.18205) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6e340000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6e310000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6e220000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6e210000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 77490000 0019d000 C:\Windows\system32\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75c40000 00027000 C:\Windows\system32\CFGMGR32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

Get Product Name Failed.
ModLoad: 77950000 0008f000 C:\Windows\system32\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded)
    Linked PDB Filename   : oleaut32.pdb
    File Version          : 6.1.7601.17676
    Company Name          : Microsoft Corporation
    Product Name          : 
    Product Version       : 6.1.7601.17676

ModLoad: 77a00000 0015c000 C:\Windows\system32\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75ed0000 00012000 C:\Windows\system32\DEVOBJ.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 72e70000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : dwmapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 76300000 0002a000 C:\Windows\system32\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 6.1.7601.17787 (win7sp1_gdr.120229-1502)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17787

ModLoad: 779e0000 0001f000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 760c0000 000cc000 C:\Windows\system32\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 5e0e0000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.8.0004.0 (debuggers(dbg).070515-1751)
    Company Name          : Microsoft Corporation
    Product Name          : Debugging Tools for Windows(R)
    Product Version       : 6.8.0004.0

ModLoad: 6caf0000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : symsrv.pdb
    File Version          : 6.8.0004.0 (debuggers(dbg).070515-1751)
    Company Name          : Microsoft Corporation
    Product Name          : Debugging Tools for Windows(R)
    Product Version       : 6.8.0004.0

ModLoad: 6dbf0000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : srcsrv.pdb
    File Version          : 6.8.0004.0 (debuggers(dbg).070515-1751)
    Company Name          : Microsoft Corporation
    Product Name          : Debugging Tools for Windows(R)
    Product Version       : 6.8.0004.0

ModLoad: 75180000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 10772 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x75C8812F

- Registers -
eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=75c8812f esp=1132f9c4 ebp=1132fa14
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000212

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 KERNELBASE!RaiseException+0x0 
1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad8 00408130 1132faec 000e23b1 1132faec 000e2339 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff88 772eed5c 7ffdd000 1132ffd4 77bc37eb 7ffdd000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 77bc37eb 7ffdd000 670c0de5 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
1132ffd4 77bc37be 007bde7f 7ffdd000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
1132ffec 00000000 007bde7f 7ffdd000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 10536 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=3266ff48 edi=00000000
eip=77ba70f4 esp=3266ff04 ebp=3266ff6c
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
3266ff6c 75c81818 00000064 00000000 3266ff88 00749924 ntdll!KiFastSystemCallRet+0x0 
3266ff7c 00749924 00000064 3266ff94 772eed5c 00000000 KERNELBASE!Sleep+0x0 
3266ff88 772eed5c 00000000 3266ffd4 77bc37eb 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 
3266ff94 77bc37eb 00000000 44580de5 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
3266ffd4 77bc37be 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
3266ffec 00000000 00749910 00000000 00000000 32b80000 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Sep 2013 10:16:01 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 233,280 224,011 0.9603
25 Sep 2013 09:27:20 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 207,360 201,761 0.9730
25 Sep 2013 09:23:35 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 181,440 176,005 0.9700
25 Sep 2013 09:14:15 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 155,520 152,185 0.9786
23 Sep 2013 11:16:55 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 129,600 127,491 0.9837
23 Sep 2013 11:16:55 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 103,680 101,557 0.9795
23 Sep 2013 11:16:55 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 77,760 76,009 0.9775
23 Sep 2013 11:16:55 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 51,840 50,796 0.9799
23 Sep 2013 11:16:55 1294177 16024975 hadcm3n_84du_1980_40_008463638_0 25,920 25,398 0.9799


©2024 climateprediction.net