climateprediction.net home page
Task 21373652

Task 21373652

Name wah2_global_e19e_208812_145_770_011666006_1
Workunit 11666006
Created 10 Nov 2018, 0:17:46 UTC
Sent 10 Nov 2018, 2:17:44 UTC
Report deadline 23 Oct 2019, 7:37:44 UTC
Received 14 Nov 2018, 20:24:14 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1469652
Run time 1 days 11 hours 3 min 25 sec
CPU time 1 days 10 hours 49 min 20 sec
Validate state Invalid
Credit 3,044.17
Device peak FLOPS 2.73 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 238.17 MB
Peak swap size 174.34 MB
Peak disk usage 252.77 MB
Stderr 
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x00463007 read attempt to address 0x012FD3AC

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 11/09/18 23:00:13
Install Directory : C:\Program Files (x86)\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\Program Files (x86)\BOINC\\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\Program Files (x86)\BOINC\\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files (x86)\BOINC\\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files (x86)\BOINC\\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\2;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 C:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077210000 0000000000190000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.254) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.17134.228 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.228

ModLoad: 0000000074680000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.17134.228 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.228

ModLoad: 0000000076bb0000 00000000001e4000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.165) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.17134.228 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.228

ModLoad: 0000000074760000 000000000018d000 C:\WINDOWS\System32\USER32.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000073d30000 0000000000017000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : wwin32u.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000074590000 0000000000022000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.17134.285 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.285

ModLoad: 0000000075fc0000 0000000000164000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.285) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32full.pdb
    File Version          : 10.0.17134.285 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.285

ModLoad: 0000000076b30000 000000000007d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000076f20000 000000000011e000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.254) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.17134.254 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.254

ModLoad: 0000000076640000 0000000000078000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000073eb0000 00000000000bf000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.17134.1

ModLoad: 00000000768f0000 0000000000044000 C:\WINDOWS\System32\sechost.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 00000000745c0000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000073ad0000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000073ac0000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000076710000 0000000000058000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.137) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.17134.137 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.137

ModLoad: 00000000702e0000 0000000000029000 C:\WINDOWS\SYSTEM32\ntmarta.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntmarta.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000068f50000 0000000000182000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 00000000738d0000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 25851, Write: 0, Other 109630

- I/O Transfers Counters -
Read: 0, Write: 3404, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 195880, QuotaPeakPagedPoolUsage: 196056
QuotaNonPagedPoolUsage: 8344, QuotaPeakNonPagedPoolUsage: 9176

- Virtual Memory Usage -
VirtualSize: 105734144, PeakVirtualSize: 107307008

- Pagefile Usage -
PagefileUsage: 2871296, PeakPagefileUsage: 3121152

- Working Set Size -
WorkingSetSize: 79372288, PeakWorkingSetSize: 79605760, PageFaultCount: 20529

*** Dump of thread ID 6008 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 3013463.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x00463007 read attempt to address 0x012FD3AC

- Registers -
eax=00000015 ebx=fffffffe ecx=00000000 edx=00000000 esi=0122d020 edi=00000000
eip=00463007 esp=0092fe7c ebp=0092fe80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010216

- Callstack -
ChildEBP RetAddr  Args to Child
0092fe80 0044826f 0122d020 00000000 0122d020 fffffffe wah2_8.24_windows_intelx86!+0x0 
0092fe98 0044fa68 0122d020 00000000 0051d6b0 0092fee8 wah2_8.24_windows_intelx86!+0x0 
0092feac 004856fc 00000015 0e8b0a12 00000006 0048566c wah2_8.24_windows_intelx86!+0x0 
0092fee8 76d02039 00000006 d8e6887e 76d01e80 76d01e80 wah2_8.24_windows_intelx86!+0x0 
0092ff80 74698484 80000006 74698460 82438746 0092ffdc KERNELBASE!CtrlRoutine+0x0 
0092ff94 7727305a 80000006 2d566810 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0092ffdc 7727302a ffffffff 7728ec9a 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 
0092ffec 00000000 76d01e80 80000006 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 

*** Dump of thread ID 3804 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 37343752.000000, User Time: 86562496.000000, Wait Time: 3013463.000000

- Registers -
eax=00000000 ebx=00000041 ecx=00000000 edx=00000000 esi=00000000 edi=0000084d
eip=7727a5ac esp=0019ed38 ebp=0019ed9c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
0019ed9c 76cb369f 000000c8 00000000 0019edc0 00468ec1 ntdll!NtDelayExecution+0x0 
0019edac 00468ec1 000000c8 000000c8 00000000 0019edf8 KERNELBASE!Sleep+0x0 
0019edc0 00407a89 a0000000 027f9999 00000000 004d0c50 wah2_8.24_windows_intelx86!+0x0 
0019edf8 004092c5 00000000 0e0009ba 00485cce 00000000 wah2_8.24_windows_intelx86!+0x0 
0019fd40 004094b7 0019fef0 00409535 004d1370 0019fd60 wah2_8.24_windows_intelx86!+0x0 
0019fd48 00409535 004d1370 0019fd60 005b32c8 0019fd60 wah2_8.24_windows_intelx86!+0x0 
0019fef0 00485c7b 00400000 00000000 005b3306 0000000a wah2_8.24_windows_intelx86!+0x0 
0019ff80 74698484 002d1000 74698460 82c88746 0019ffdc wah2_8.24_windows_intelx86!+0x0 
0019ff94 7727305a 002d1000 2ddd6810 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0019ffdc 7727302a ffffffff 7728ec9a 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 
0019ffec 00000000 00485cce 002d1000 00000000 78746341 ntdll!RtlValidSecurityDescriptor+0x0 

*** Dump of thread ID 11000 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 3013463.000000

- Registers -
eax=c0000022 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000001
eip=7727a5ac esp=0121fcdc ebp=0121fd40
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
0121fd40 76cb369f 00000064 00000000 0121ff80 00478b44 ntdll!NtDelayExecution+0x0 
0121fd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 
0121ff80 74698484 00000000 74698460 83f08746 0121ffdc wah2_8.24_windows_intelx86!+0x0 
0121ff94 7727305a 00000000 2ce56810 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
0121ffdc 7727302a ffffffff 7728ec9a 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 
0121ffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlValidSecurityDescriptor+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7208, selfPID=1180, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7208, selfPID=7208, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3580, selfPID=4480, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3580, selfPID=3580, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11680, selfPID=12132, iMonCtr=1
No Process Handle
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11680, selfPID=11680, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
BUFFIN: C I/O Error feof - Unit 62 - Return code = 16

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/xadae.pipe_dummy                                                            
Leaving CPDN_ain::Monitor...
14:23:22 (4348): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_17.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_18.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_19.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_20.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_21.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_22.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_23.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_24.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_25.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_26.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_27.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_28.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_29.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_30.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_31.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_32.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_33.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_34.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_35.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_36.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_37.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_38.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_39.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_40.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_41.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_42.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_43.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_44.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_45.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_46.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_47.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_48.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_49.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_50.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_51.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_52.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_53.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_54.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_55.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_56.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_57.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_58.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_59.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_60.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_61.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_62.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_63.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_64.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_65.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_66.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_67.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_68.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_69.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_70.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_71.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_72.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_73.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_74.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_75.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_76.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_77.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_78.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_79.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_80.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_81.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_82.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_83.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_84.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_85.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_86.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_87.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_88.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_89.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_90.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_91.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_92.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_93.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_94.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_95.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_96.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_97.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_98.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_99.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_100.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_101.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_102.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_103.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_104.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_105.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_106.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_107.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_108.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_109.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_110.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_111.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_112.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_113.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_114.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_115.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_116.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_117.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_118.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_119.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_120.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_121.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_122.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_123.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_124.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_125.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_126.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_127.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_128.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_129.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_130.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_131.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_132.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_133.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_134.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_135.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_136.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_137.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_138.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_139.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_140.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_141.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_142.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_143.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_144.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_145.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>wah2_global_e19e_208812_145_770_011666006_1_r745945361_restart.zip</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Nov 2018 19:16:56 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 46,147 125,275 2.7147
14 Nov 2018 17:08:32 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 43,267 117,456 2.7147
14 Nov 2018 14:59:43 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 40,387 109,862 2.7202
14 Nov 2018 12:56:48 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 37,507 102,429 2.7309
14 Nov 2018 10:53:27 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 34,627 94,984 2.7431
14 Nov 2018 08:44:33 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 31,747 87,459 2.7549
14 Nov 2018 06:36:50 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 28,867 79,883 2.7673
14 Nov 2018 04:28:59 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 25,987 72,280 2.7814
14 Nov 2018 02:25:49 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 23,107 64,692 2.7997
13 Nov 2018 02:46:57 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 20,227 56,832 2.8097
13 Nov 2018 00:15:20 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 17,347 48,858 2.8165
11 Nov 2018 02:37:46 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 14,467 40,885 2.8261
11 Nov 2018 00:20:27 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 11,587 32,838 2.8340
10 Nov 2018 22:08:48 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 8,707 24,829 2.8516
10 Nov 2018 19:47:18 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 5,827 16,527 2.8363
10 Nov 2018 14:56:23 1469652 21373652 wah2_global_e19e_208812_145_770_011666006_1 2,947 8,283 2.8107


©2024 cpdn.org