climateprediction.net home page
Task 16445186

Task 16445186

Name hadam3p_anz_a623_2012_1_008614187_0
Workunit 8760699
Created 2 Apr 2014, 14:41:19 UTC
Sent 30 Apr 2014, 2:43:59 UTC
Report deadline 12 Apr 2015, 8:03:59 UTC
Received 7 May 2014, 2:36:43 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1007570
Run time 1 days 20 hours 35 min 11 sec
CPU time 1 days 20 hours 19 min 48 sec
Validate state Invalid
Credit 1,006.54
Device peak FLOPS 3.04 GFLOPS
Application version UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10
windows_intelx86
Stderr 
<core_client_version>7.2.42</core_client_version>
<![CDATA[
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
15:58:13 (4396): No heartbeat from core client for 30 sec - exiting
15:58:14 (4396): No heartbeat from core client for 30 sec - exiting
15:58:15 (4396): No heartbeat from core client for 30 sec - exiting
15:58:16 (4396): No heartbeat from core client for 30 sec - exiting
15:58:17 (4396): No heartbeat from core client for 30 sec - exiting
15:58:19 (4396): No heartbeat from core client for 30 sec - exiting
15:58:20 (4396): No heartbeat from core client for 30 sec - exiting
15:58:21 (4396): No heartbeat from core client for 30 sec - exiting
15:58:22 (4396): No heartbeat from core client for 30 sec - exiting
15:58:23 (4396): No heartbeat from core client for 30 sec - exiting
15:58:24 (4396): No heartbeat from core client for 30 sec - exiting
15:58:25 (4396): No heartbeat from core client for 30 sec - exiting
15:58:26 (4396): No heartbeat from core client for 30 sec - exiting
15:58:27 (4396): No heartbeat from core client for 30 sec - exiting
15:58:28 (4396): No heartbeat from core client for 30 sec - exiting
15:58:30 (4396): No heartbeat from core client for 30 sec - exiting
15:58:31 (4396): No heartbeat from core client for 30 sec - exiting
15:58:32 (4396): No heartbeat from core client for 30 sec - exiting
15:58:33 (4396): No heartbeat from core client for 30 sec - exiting
15:58:34 (4396): No heartbeat from core client for 30 sec - exiting
15:58:35 (4396): No heartbeat from core client for 30 sec - exiting
15:58:36 (4396): No heartbeat from core client for 30 sec - exiting
15:58:37 (4396): No heartbeat from core client for 30 sec - exiting
15:58:38 (4396): No heartbeat from core client for 30 sec - exiting
15:58:39 (4396): No heartbeat from core client for 30 sec - exiting
15:58:40 (4396): No heartbeat from core client for 30 sec - exiting
15:58:42 (4396): No heartbeat from core client for 30 sec - exiting
15:58:43 (4396): No heartbeat from core client for 30 sec - exiting
15:58:44 (4396): No heartbeat from core client for 30 sec - exiting
15:58:45 (4396): No heartbeat from core client for 30 sec - exiting
15:58:46 (4396): No heartbeat from core client for 30 sec - exiting
15:58:47 (4396): No heartbeat from core client for 30 sec - exiting
15:58:48 (4396): No heartbeat from core client for 30 sec - exiting
15:58:49 (4396): No heartbeat from core client for 30 sec - exiting
15:58:50 (4396): No heartbeat from core client for 30 sec - exiting
15:58:51 (4396): No heartbeat from core client for 30 sec - exiting
15:58:52 (4396): No heartbeat from core client for 30 sec - exiting
15:58:54 (4396): No heartbeat from core client for 30 sec - exiting
15:58:55 (4396): No heartbeat from core client for 30 sec - exiting
15:58:56 (4396): No heartbeat from core client for 30 sec - exiting
15:58:57 (4396): No heartbeat from core client for 30 sec - exiting
15:58:58 (4396): No heartbeat from core client for 30 sec - exiting
15:58:59 (4396): No heartbeat from core client for 30 sec - exiting
15:59:00 (4396): No heartbeat from core client for 30 sec - exiting
15:59:01 (4396): No heartbeat from core client for 30 sec - exiting
15:59:02 (4396): No heartbeat from core client for 30 sec - exiting
15:59:03 (4396): No heartbeat from core client for 30 sec - exiting
15:59:04 (4396): No heartbeat from core client for 30 sec - exiting
15:59:06 (4396): No heartbeat from core client for 30 sec - exiting
15:59:07 (4396): No heartbeat from core client for 30 sec - exiting
15:59:08 (4396): No heartbeat from core client for 30 sec - exiting
15:59:09 (4396): No heartbeat from core client for 30 sec - exiting
15:59:10 (4396): No heartbeat from core client for 30 sec - exiting
15:59:11 (4396): No heartbeat from core client for 30 sec - exiting
15:59:12 (4396): No heartbeat from core client for 30 sec - exiting
15:59:13 (4396): No heartbeat from core client for 30 sec - exiting
15:59:14 (4396): No heartbeat from core client for 30 sec - exiting
15:59:15 (4396): No heartbeat from core client for 30 sec - exiting
15:59:16 (4396): No heartbeat from core client for 30 sec - exiting
15:59:18 (4396): No heartbeat from core client for 30 sec - exiting
15:59:19 (4396): No heartbeat from core client for 30 sec - exiting
15:59:20 (4396): No heartbeat from core client for 30 sec - exiting
15:59:21 (4396): No heartbeat from core client for 30 sec - exiting
15:59:22 (4396): No heartbeat from core client for 30 sec - exiting
15:59:23 (4396): No heartbeat from core client for 30 sec - exiting
15:59:24 (4396): No heartbeat from core client for 30 sec - exiting
15:59:25 (4396): No heartbeat from core client for 30 sec - exiting
15:59:26 (4396): No heartbeat from core client for 30 sec - exiting
15:59:27 (4396): No heartbeat from core client for 30 sec - exiting
15:59:28 (4396): No heartbeat from core client for 30 sec - exiting
15:59:30 (4396): No heartbeat from core client for 30 sec - exiting
15:59:31 (4396): No heartbeat from core client for 30 sec - exiting
15:59:32 (4396): No heartbeat from core client for 30 sec - exiting
15:59:33 (4396): No heartbeat from core client for 30 sec - exiting
15:59:34 (4396): No heartbeat from core client for 30 sec - exiting
15:59:35 (4396): No heartbeat from core client for 30 sec - exiting
15:59:36 (4396): No heartbeat from core client for 30 sec - exiting
15:59:37 (4396): No heartbeat from core client for 30 sec - exiting
15:59:38 (4396): No heartbeat from core client for 30 sec - exiting
15:59:39 (4396): No heartbeat from core client for 30 sec - exiting
15:59:40 (4396): No heartbeat from core client for 30 sec - exiting
15:59:41 (4396): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9176, selfPID=9176, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=20772, selfPID=20772, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Could not create shared memory region hadam3p_anz_a623_2012_1_008614187, 30917504
Error in creating shared memory region!
Called boinc_finish


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x762CC41F

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 05/07/14 04:27:12
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadam3p_anz_a623_2012_1_008614187;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00eb0000 008e6000 C:\ProgramData\BOINC\projects\climateprediction.net\hadrm3p_anz_um_6.10_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : d:\cpdn\cpdnboinc\cpdnprecis\test\projects\reqs.comlab.ox.ac.uk_cpdn\hadrm3p_anz_um_6.10_windows_intelx86.pdb

ModLoad: 77ad0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.18247) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 764f0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.18409) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.18409 (win7sp1_gdr.140303-2144)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18409

ModLoad: 762c0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.18229) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.18229 (win7sp1_gdr.130801-1533)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18229

ModLoad: 755a0000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 75880000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.18275) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.18275 (win7sp1_gdr.131002-1533)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18275

ModLoad: 75ca0000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.18177) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7601.18177 (win7sp1_gdr.130605-1534)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18177

ModLoad: 75f50000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.18009) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.18009 (win7sp1_gdr.121121-1431)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.18009

ModLoad: 759f0000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 76600000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.18247) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 76a10000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 766a0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.18205) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 75500000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.18270) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.18270 (win7sp1_gdr.130924-1532)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18270

ModLoad: 754f0000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 29804 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x762CC41F

- Registers -
eax=12f2f34c ebx=00000001 ecx=00000003 edx=00000000 esi=01342428 edi=fffffffe
eip=762cc41f esp=12f2f34c ebp=12f2f39c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
12f2f39c 011d1d43 e06d7363 00000001 00000003 12f2f3c8 KERNELBASE!RaiseException+0x0 
12f2f3d4 011cef45 12f2f3e4 01269608 0123f15c 01265754 hadrm3p_anz_um_6.10_windows_int!+0x0 
12f2f3f0 00eb8352 01000000 00eb98a8 00000000 00b60000 hadrm3p_anz_um_6.10_windows_int!+0x0 
12f2f45c 00eb95e0 12f2f470 002438e8 12f2f470 00243869 hadrm3p_anz_um_6.10_windows_int!+0x0 
12f2f90c 7650338a 7efde000 12f2f958 77b09f72 7efde000 hadrm3p_anz_um_6.10_windows_int!+0x0 
12f2f918 77b09f72 7efde000 6fa6929b 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
12f2f958 77b09f45 011d0ea6 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
12f2f970 00000000 011d0ea6 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 15424 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=0115bb20 ebx=00000000 ecx=00000000 edx=00000000 esi=390ef8b8 edi=00000000
eip=77aefd91 esp=390ef874 ebp=390ef8dc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
390ef8dc 762d4498 00000064 00000000 390ef8f8 0115bb34 ntdll!ZwDelayExecution+0x0 
390ef8ec 0115bb34 00000064 390ef904 7650338a 00000000 KERNELBASE!Sleep+0x0 
390ef8f8 7650338a 00000000 390ef944 77b09f72 00000000 hadrm3p_anz_um_6.10_windows_int!+0x0 
390ef904 77b09f72 00000000 445a9287 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
390ef944 77b09f45 0115bb20 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
390ef95c 00000000 0115bb20 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=25844, iMonCtr=2
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Called boinc_finish

</stderr_txt><message>
upload failure: <file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_3.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_4.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_5.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_6.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_7.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_8.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_9.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_10.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_11.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_12.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>hadam3p_anz_a623_2012_1_008614187_0_13.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>

</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
06 May 2014 04:40:30 1007570 16445186 hadam3p_anz_a623_2012_1_008614187_0 23,339 121,802 5.2188
02 May 2014 17:31:18 1007570 16445186 hadam3p_anz_a623_2012_1_008614187_0 11,819 61,907 5.2379


©2024 climateprediction.net