climateprediction.net home page
Task 21415235

Task 21415235

Name wah2_pnw25_cc36_190312_49_695_011385712_0
Workunit 11385712
Created 10 Dec 2018, 11:49:07 UTC
Sent 10 Dec 2018, 11:49:20 UTC
Report deadline 22 Nov 2019, 17:09:20 UTC
Received 19 Dec 2018, 21:18:10 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID 1348018
Run time 8 days 0 hours 12 min 51 sec
CPU time 6 days 11 hours 6 min 35 sec
Validate state Invalid
Credit 29,657.24
Device peak FLOPS 4.20 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 187.22 MB
Peak swap size 166.02 MB
Peak disk usage 7.08 MB
Stderr
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
finish file present too long</message>
<stderr_txt>
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda3dec
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4feb
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4mar
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4apr
CPDN Monitor - Quit request from BOINC...
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4may
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4jun
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4jul
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4aug
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4sep
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4oct
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4nov
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda4dec
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5feb
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5mar
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5apr
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5may
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5jun
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5jul
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5aug
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5sep
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5oct
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5nov
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda5dec
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6feb
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6mar
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6apr
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6may
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6jun
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6jul
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6aug
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6sep
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6oct
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6nov
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda6dec
Error converting file to netcdf: F:\BOINC/projects/climateprediction.net/wah2_pnw25_cc36_190312_49_695_011385712/dataout\cc36ga.pda7feb
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16212, selfPID=8876, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_ain::Monitor...
13:00:13 (8876): called boinc_finish(0)
CPDN Monitor - Abort request from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0274812C

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 12/19/18 13:02:53
Install Directory : 
Data Directory    : F:\BOINC
Project Symstore  : 
LoadLibraryA( F:\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( F:\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( F:\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( F:\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: F:\BOINC\projects\climateprediction.net;F:\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 F:\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077360000 0000000000190000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.17134.228 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.228

ModLoad: 0000000073fe0000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.17134.376 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.376

ModLoad: 0000000074de0000 00000000001e4000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.441) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.17134.376 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.376

ModLoad: 00000000740c0000 000000000018d000 C:\WINDOWS\System32\USER32.dll (6.2.17134.376) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000076ee0000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : wwin32u.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000074d10000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.17134.285 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.285

ModLoad: 0000000076f50000 0000000000164000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32full.pdb
    File Version          : 10.0.17134.471 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.471

ModLoad: 00000000770c0000 000000000007d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000076c20000 000000000011e000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.319) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.17134.319 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.319

ModLoad: 00000000771f0000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000073f20000 00000000000bf000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.17134.1

ModLoad: 00000000755d0000 0000000000044000 C:\WINDOWS\System32\sechost.dll (6.2.17134.319) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000074480000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000073c20000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17134.376) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.17134.376 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.376

ModLoad: 0000000073c10000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000074420000 0000000000058000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.17134.471 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.471

ModLoad: 0000000075400000 0000000000026000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000006d680000 0000000000182000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000073950000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 4021, Write: 0, Other 52799

- I/O Transfers Counters -
Read: 0, Write: 1682, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 91944, QuotaPeakPagedPoolUsage: 92104
QuotaNonPagedPoolUsage: 8488, QuotaPeakNonPagedPoolUsage: 9048

- Virtual Memory Usage -
VirtualSize: 55767040, PeakVirtualSize: 56291328

- Pagefile Usage -
PagefileUsage: 3735552, PeakPagefileUsage: 3825664

- Working Set Size -
WorkingSetSize: 6447104, PeakWorkingSetSize: 6471680, PageFaultCount: 1902

*** Dump of thread ID 7500 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 12343750.000000, User Time: 11875000.000000, Wait Time: 6787.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0274812C

- Registers -
eax=00000065 ebx=004b63b9 ecx=0019dc88 edx=0274812c esi=0019da40 edi=0019dcb8
eip=0048ee46 esp=0019d9d4 ebp=0019dc64
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202

- Callstack -
ChildEBP RetAddr  Args to Child
0019dc64 0048161c 0019dc88 004b63b8 00000000 0019dcb8 wah2_8.24_windows_intelx86!+0x0 
0019dca8 0044fa52 0274812c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 
0019dcc8 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0019dce0 00485ca1 c0000005 0019dd14 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 
0019ff80 73ff8484 00380000 73ff8460 cb10dcc8 0019ffdc wah2_8.24_windows_intelx86!+0x0 
0019ff94 773c3ab8 00380000 ed92e395 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0019ffdc 773c3a88 ffffffff 773df31a 00000000 00000000 ntdll!RtlAreBitsSet+0x0 
0019ffec 00000000 00485cce 00380000 00000000 78746341 ntdll!RtlAreBitsSet+0x0 

*** Dump of thread ID 936 (state: Running): ***

- Information -
Status: Base Priority: Unknown, Priority: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6788.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000000
eip=00000000 esp=00000000 ebp=00000000
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000

- Callstack -
ChildEBP RetAddr  Args to Child
(-nosymbols- PC == 0)
00000000 00000000 00000000 00000000 00000000 00000000 !reBitsSet+0x0 

*** Dump of thread ID 3540 (state: Ready): ***

- Information -
Status: Base Priority: Unknown, Priority: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6788.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000000
eip=00000000 esp=00000000 ebp=00000000
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000

- Callstack -
ChildEBP RetAddr  Args to Child
(-nosymbols- PC == 0)
00000000 00000000 00000000 00000000 00000000 00000000 !reBitsSet+0x0 

*** Dump of thread ID 16188 (state: Running): ***

- Information -
Status: Base Priority: Unknown, Priority: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6788.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000000
eip=00000000 esp=00000000 ebp=00000000
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000

- Callstack -
ChildEBP RetAddr  Args to Child
(-nosymbols- PC == 0)
00000000 00000000 00000000 00000000 00000000 00000000 !reBitsSet+0x0 

*** Dump of thread ID 4716 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6786.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001
eip=773cac1c esp=0267fcdc ebp=0267fd40
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
0267fd40 74ee70ef 00000064 00000000 0267ff80 00478b44 ntdll!NtDelayExecution+0x0 
0267fd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 
0267ff80 73ff8484 00000000 73ff8460 c96edcc8 0267ffdc wah2_8.24_windows_intelx86!+0x0 
0267ff94 773c3ab8 00000000 efece395 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0267ffdc 773c3a88 ffffffff 773df31a 00000000 00000000 ntdll!RtlAreBitsSet+0x0 
0267ffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlAreBitsSet+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
19 Dec 2018 21:22:16 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 449,579 554,992 1.2345
19 Dec 2018 21:22:16 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 438,059 540,759 1.2344
19 Dec 2018 21:22:16 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 426,539 526,721 1.2349
19 Dec 2018 21:22:16 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 415,019 512,601 1.2351
19 Dec 2018 02:57:06 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 403,499 498,228 1.2348
18 Dec 2018 19:46:25 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 391,979 483,518 1.2335
18 Dec 2018 14:16:55 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 380,459 469,308 1.2335
18 Dec 2018 09:45:34 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 368,939 454,603 1.2322
18 Dec 2018 03:56:53 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 357,419 439,720 1.2303
17 Dec 2018 22:49:39 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 345,899 425,242 1.2294
17 Dec 2018 16:42:58 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 334,379 410,066 1.2264
17 Dec 2018 11:30:33 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 322,859 395,853 1.2261
17 Dec 2018 06:25:22 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 311,339 381,709 1.2260
17 Dec 2018 02:29:18 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 299,819 367,543 1.2259
16 Dec 2018 19:38:18 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 288,299 353,150 1.2249
16 Dec 2018 14:24:11 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 276,779 338,930 1.2246
16 Dec 2018 09:57:22 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 265,259 325,067 1.2255
16 Dec 2018 05:47:18 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 253,739 311,355 1.2271
16 Dec 2018 00:55:38 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 242,219 297,472 1.2281
15 Dec 2018 20:28:03 1348018 21415235 wah2_pnw25_cc36_190312_49_695_011385712_0 230,699 283,612 1.2294


©2024 climateprediction.net