Task 21177955

Name	wah2_nam50_p7lc_198912_13_735_011558454_0
Workunit	11558454
Created	17 Jul 2018, 11:57:15 UTC
Sent	17 Jul 2018, 17:49:37 UTC
Report deadline	29 Jun 2019, 23:09:37 UTC
Received	23 Jul 2018, 4:03:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID	1323201
Run time	17 hours 10 min 15 sec
CPU time	14 hours 44 min 10 sec
Validate state	Invalid
Credit	1,539.60
Device peak FLOPS	3.98 GFLOPS
Application version	Weather At Home 2 (wah2) v8.24 windows_intelx86
Peak working set size	202.14 MB
Peak swap size	170.75 MB
Peak disk usage	36.59 MB
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> finish file present too long </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=680, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_ain::Monitor... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/wah2_nam50_p7lc_198912_13_735_011558454/dataout/atmos_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5720, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_ain::Monitor... 21:31:44 (5720): called boinc_finish(0) CPDN Monitor - Abort request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0254012C Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 07/22/18 23:03:13 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000152000 C:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077720000 0000000000180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.24168) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 0000000076ce0000 0000000000110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.24168) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 0000000077070000 0000000000047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.24168) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 00000000757d0000 0000000000100000 C:\Windows\syswow64\USER32.dll (6.1.7601.23594) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 0000000075490000 0000000000090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.23914) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7601.23914 (win7sp1_ldr.170908-0600) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.23914 ModLoad: 0000000076860000 000000000000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.24082) (-exported- Symbols Loaded) Linked PDB Filename : wlpk.pdb File Version : 6.1.7601.24082 (win7sp1_ldr.180309-0600) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.24082 ModLoad: 0000000076e80000 000000000009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.23894) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.7601.23894 (win7sp1_ldr.170816-0600) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.7601.23894 ModLoad: 0000000076870000 00000000000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.7601.17744 (win7sp1_gdr.111215-1535) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.7601.17744 ModLoad: 0000000076a70000 00000000000a1000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.24168) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.1.7601.24168 (win7sp1_ldr.180608-0600) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.24168 ModLoad: 00000000752c0000 0000000000019000 C:\Windows\SysWOW64\sechost.dll (6.1.7601.18869) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 0000000076f20000 00000000000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.24168) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 0000000074fd0000 0000000000060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.24168) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.1.7601.24168 (win7sp1_ldr.180608-0600) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.24168 ModLoad: 0000000074fc0000 000000000000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7601.24168) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.1.7601.24168 (win7sp1_ldr.180608-0600) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.24168 ModLoad: 0000000076970000 0000000000060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 0000000075b20000 00000000000cd000 C:\Windows\syswow64\MSCTF.dll (6.1.7601.23915) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 0000000060910000 00000000000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 0000000074fa0000 0000000000009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 6397, Write: 0, Other 35799 - I/O Transfers Counters - Read: 0, Write: 4168, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 89872, QuotaPeakPagedPoolUsage: 89872 QuotaNonPagedPoolUsage: 6720, QuotaPeakNonPagedPoolUsage: 7400 - Virtual Memory Usage - VirtualSize: 51040256, PeakVirtualSize: 51896320 - Pagefile Usage - PagefileUsage: 2580480, PeakPagefileUsage: 2875392 - Working Set Size - WorkingSetSize: 4997120, PeakWorkingSetSize: 5206016, PageFaultCount: 1463 * Dump of thread ID 5156 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 4368028.000000, User Time: 11388073.000000, Wait Time: 15592895.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0254012C - Registers - eax=00000065 ebx=004b63b9 ecx=0018dca8 edx=0254012c esi=0018da60 edi=0018dcd8 eip=0048ee46 esp=0018d9f4 ebp=0018dc84 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 - Callstack - ChildEBP RetAddr Args to Child 0018dc84 0048161c 0018dca8 004b63b8 00000000 0018dcd8 wah2_8.24_windows_intelx86!+0x0 0018dcc8 0044fa52 0254012c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 0018dce8 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 0018dd00 00485ca1 c0000005 0018dd34 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 0018ff88 76cf343d 7efde000 0018ffd4 77759832 7efde000 wah2_8.24_windows_intelx86!+0x0 0018ff94 77759832 7efde000 79e55242 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 0018ffd4 77759805 00485cce 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 0018ffec 00000000 00485cce 7efde000 00000000 78746341 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 7844 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 15592896.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=0247fd24 edi=00000000 eip=7773fdc1 esp=0247fce0 ebp=0247fd48 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 0247fd48 7708460c 00000064 00000000 0247ff88 00478b44 ntdll!NtDelayExecution+0x0 0247fd58 00478b44 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 0247ff88 76cf343d 00000000 0247ffd4 77759832 00000000 wah2_8.24_windows_intelx86!+0x0 0247ff94 77759832 00000000 7bba5242 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 0247ffd4 77759805 00478b20 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 0247ffec 00000000 00478b20 00000000 00000000 5c319267 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Jul 2018 04:41:09	1323201	21177955	wah2_nam50_p7lc_198912_13_735_011558454_0	23,339	43,597	1.8680
18 Jul 2018 01:40:09	1323201	21177955	wah2_nam50_p7lc_198912_13_735_011558454_0	11,819	21,912	1.8540