Task 20480406

Name	wah2_wus25_tjdt_200409_25_583_011072429_1
Workunit	11072429
Created	9 Jun 2017, 3:26:02 UTC
Sent	9 Jun 2017, 3:49:19 UTC
Report deadline	22 May 2018, 9:09:19 UTC
Received	13 Jun 2017, 3:30:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID	1382405
Run time	18 hours 19 min 26 sec
CPU time	18 hours 19 min 26 sec
Validate state	Invalid
Credit	2,299.53
Device peak FLOPS	3.65 GFLOPS
Application version	Weather At Home 2 (wah2) v8.24 windows_intelx86
Peak working set size	182.11 MB
Peak swap size	165.13 MB
Peak disk usage	0.02 MB
Stderr	<core_client_version>7.6.33</core_client_version> <![CDATA[ <message> finish file present too long </message> <stderr_txt> 18:14:06 (13128): start_timer_thread(): CreateThread() failed, errno 0 18:14:07 (21264): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:13:09 (22596): start_timer_thread(): CreateThread() failed, errno 0 19:13:10 (1724): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:27:47 (20064): start_timer_thread(): CreateThread() failed, errno 0 19:27:49 (18948): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:36:08 (6468): start_timer_thread(): CreateThread() failed, errno 0 19:36:09 (22144): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:46:52 (17096): start_timer_thread(): CreateThread() failed, errno 0 19:46:54 (11716): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:08:00 (12792): start_timer_thread(): CreateThread() failed, errno 0 20:08:01 (10060): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:22:57 (23440): start_timer_thread(): CreateThread() failed, errno 0 20:22:58 (13296): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:45:15 (7372): start_timer_thread(): CreateThread() failed, errno 0 20:45:17 (17748): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:45:09 (25960): start_timer_thread(): CreateThread() failed, errno 0 23:45:11 (25932): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:46:03 (26488): start_timer_thread(): CreateThread() failed, errno 0 02:46:05 (8928): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:01:39 (35460): start_timer_thread(): CreateThread() failed, errno 0 07:01:41 (35552): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:29:58 (11052): start_timer_thread(): CreateThread() failed, errno 0 07:30:00 (35928): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:44:55 (35660): start_timer_thread(): CreateThread() failed, errno 0 07:44:57 (33244): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:45:40 (22836): start_timer_thread(): CreateThread() failed, errno 0 08:45:42 (34388): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:45:51 (30488): start_timer_thread(): CreateThread() failed, errno 0 11:45:52 (26968): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 12:17:50 (25864): start_timer_thread(): CreateThread() failed, errno 0 12:17:52 (34796): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:45:47 (43020): start_timer_thread(): CreateThread() failed, errno 0 14:45:48 (41936): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:01:31 (44424): start_timer_thread(): CreateThread() failed, errno 0 16:01:33 (44172): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:04:42 (6364): start_timer_thread(): CreateThread() failed, errno 0 16:04:44 (43288): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:10:51 (51380): start_timer_thread(): CreateThread() failed, errno 0 19:10:53 (44872): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:33:34 (48364): start_timer_thread(): CreateThread() failed, errno 0 19:33:36 (42240): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:51:13 (4880): start_timer_thread(): CreateThread() failed, errno 0 07:51:14 (16436): start_timer_thread(): CreateThread() failed, errno 0 09:39:29 (4912): start_timer_thread(): CreateThread() failed, errno 0 Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xadae.pipe_dummy 09:39:31 (7720): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7720, selfPID=7720, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7720, selfPID=11420, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_ain::Monitor... 09:39:37 (11420): called boinc_finish(0) CPDN Monitor - Abort request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0279E12C Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 06/10/17 09:42:49 Install Directory : Data Directory : D:\ProgramData\BOINC Project Symstore : LoadLibraryA( D:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( D:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( D:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( D:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\ProgramData\BOINC\projects\climateprediction.net;D:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000152000 D:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077ce0000 000000000018e000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000775a0000 00000000000d0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.15063.296) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000077830000 00000000001c2000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.15063.296) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000770f0000 000000000013c000 C:\WINDOWS\System32\USER32.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000776d0000 0000000000016000 C:\WINDOWS\System32\win32u.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000077800000 0000000000021000 C:\WINDOWS\System32\GDI32.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000074870000 0000000000159000 C:\WINDOWS\System32\gdi32full.dll (6.2.15063.296) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.15063.296 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.296 ModLoad: 0000000076f00000 0000000000079000 C:\WINDOWS\System32\msvcp_win.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000749d0000 0000000000118000 C:\WINDOWS\System32\ucrtbase.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000076c70000 0000000000077000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000774e0000 00000000000bd000 C:\WINDOWS\System32\msvcrt.dll (7.0.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.15063.0 ModLoad: 00000000747d0000 0000000000041000 C:\WINDOWS\System32\sechost.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000773c0000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000747b0000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000747a0000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000077b80000 0000000000057000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.15063.138) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.15063.138 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.138 ModLoad: 0000000077390000 0000000000025000 C:\WINDOWS\System32\IMM32.DLL (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000070b20000 000000000015d000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.15063.250) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.15063.250 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.250 ModLoad: 00000000744a0000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 8459, Write: 0, Other 49298 - I/O Transfers Counters - Read: 0, Write: 3765, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 90520, QuotaPeakPagedPoolUsage: 90648 QuotaNonPagedPoolUsage: 7808, QuotaPeakNonPagedPoolUsage: 9048 - Virtual Memory Usage - VirtualSize: 51826688, PeakVirtualSize: 54693888 - Pagefile Usage - PagefileUsage: 3629056, PeakPagefileUsage: 3629056 - Working Set Size - WorkingSetSize: 6311936, PeakWorkingSetSize: 6311936, PageFaultCount: 1863 * Dump of thread ID 8708 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 3906250.000000, User Time: 13906250.000000, Wait Time: 399894.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0279E12C - Registers - eax=00000065 ebx=004b63b9 ecx=0019d8e8 edx=0279e12c esi=0019d6a0 edi=0019d918 eip=0048ee46 esp=0019d634 ebp=0019d8c4 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 - Callstack - ChildEBP RetAddr Args to Child 0019d8c4 0048161c 0019d8e8 004b63b8 00000000 0019d918 wah2_8.24_windows_intelx86!+0x0 0019d908 0044fa52 0279e12c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 0019d928 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 0019d940 00485ca1 c0000005 0019d974 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 0019ff80 775b8744 00378000 775b8720 07a26e1a 0019ffdc wah2_8.24_windows_intelx86!+0x0 0019ff94 77d4587d 00378000 264a7e22 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0019ffdc 77d4584d ffffffff 77d6633d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 0019ffec 00000000 00485cce 00378000 00000000 78746341 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Dump of thread ID 6284 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 399889.000000 - Registers - eax=00000000 ebx=026cfd1c ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=77d520ac esp=026cfcdc ebp=026cfd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 026cfd40 7792351f 00000064 00000000 026cff80 00478b44 ntdll!NtDelayExecution+0x0 026cfd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 026cff80 775b8744 00000000 775b8720 05d76e1a 026cffdc wah2_8.24_windows_intelx86!+0x0 026cff94 77d4587d 00000000 243f7e22 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 026cffdc 77d4584d ffffffff 77d6633d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 026cffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Jun 2017 05:58:47	1382405	20480406	wah2_wus25_tjdt_200409_25_583_011072429_1	34,859	62,366	1.7891
09 Jun 2017 20:54:40	1382405	20480406	wah2_wus25_tjdt_200409_25_583_011072429_1	23,339	41,764	1.7895
09 Jun 2017 12:27:26	1382405	20480406	wah2_wus25_tjdt_200409_25_583_011072429_1	11,819	21,653	1.8321