Task 19407369

Name	hadam3p_afr50_eyql_201412_12_371_010406555_0
Workunit	10406555
Created	19 Mar 2016, 21:52:10 UTC
Sent	2 Apr 2016, 0:41:19 UTC
Report deadline	15 Mar 2017, 6:01:19 UTC
Received	14 Jul 2016, 17:22:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	9 (0x00000009) Unknown error code
Computer ID	1375535
Run time	4 days 9 hours 56 min 57 sec
CPU time	3 days 5 hours 35 min 58 sec
Validate state	Invalid
Credit	3,157.05
Device peak FLOPS	1.23 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Africa v7.22 windows_intelx86
Peak working set size	192.82 MB
Peak swap size	168.10 MB
Peak disk usage	27.19 MB
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> The storage control block address is invalid. (0x9) - exit code 9 (0x9) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x00462A57 read attempt to address 0x006CB3AC Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.2.4 Dump Timestamp : 04/01/16 20:28:00 Install Directory : Z:\Boinc\ Data Directory : E:\_Apps\Science\Boinc\Data Project Symstore : LoadLibraryA( Z:\Boinc\\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( Z:\Boinc\\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( Z:\Boinc\\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( Z:\Boinc\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: E:\_Apps\Science\Boinc\Data\slots\8;E:\_Apps\Science\Boinc\Data\projects\climateprediction.net ModLoad: 0000000000400000 00000000001d6000 E:\_Apps\Science\Boinc\Data\projects\climateprediction.net\hadam3p_afr_7.22_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077d90000 0000000000179000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.10240.16683) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 00000000755a0000 00000000000f0000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000075420000 0000000000176000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.2.10240.16683) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000074bb0000 000000000003e000 E:\_Utils\Security\Norton\Norton 360\NortonData\22.5.2.15\Definitions\BASHDefs\20160401.001\UMEngx86.dll (10.0.1.5) (-exported- Symbols Loaded) Linked PDB Filename : C:\bld_area\BehaviorBlocking\BB10.0\Src\10.0.1\Bin\Win32\Release\UMEngx86.pdb File Version : 10.0.1.5 Company Name : Symantec Corporation Product Name : BASH Product Version : 10.0.1.5 ModLoad: 00000000757b0000 0000000000140000 C:\WINDOWS\SYSTEM32\USER32.dll (6.2.10240.16603) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000077a70000 000000000014d000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.2.10240.16644) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.10240.16644 (th1.160104-1507) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16644 ModLoad: 0000000074d70000 000000000007b000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.2.10240.16644) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000075b70000 00000000000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.10240.16384 ModLoad: 0000000077cb0000 0000000000043000 C:\WINDOWS\SYSTEM32\sechost.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000075960000 00000000000ac000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.2.10240.16412) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000074c60000 000000000001e000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000074c50000 000000000000a000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000074bf0000 0000000000059000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 00000000756f0000 000000000002b000 C:\WINDOWS\SYSTEM32\IMM32.DLL (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000076140000 0000000000120000 C:\WINDOWS\SYSTEM32\MSCTF.dll (6.2.10240.16683) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 000000006be80000 0000000000028000 C:\WINDOWS\SYSTEM32\ntmarta.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : ntmarta.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 000000006fc60000 000000000013f000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 0000000074780000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 764 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x00462A57 read attempt to address 0x006CB3AC - Registers - eax=00000015 ebx=fffffffe ecx=00000000 edx=00000000 esi=005fb020 edi=00000000 eip=00462a57 esp=0082fe7c ebp=0082fe80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010216 - Callstack - ChildEBP RetAddr Args to Child 0082fe80 00447c3f 005fb020 00000000 005fb020 fffffffe hadam3p_afr_7.22_windows_intelx!+0x0 0082fe98 0044f438 005fb020 00000000 005a129c 0082fee8 hadam3p_afr_7.22_windows_intelx!+0x0 0082feac 00483c9c 00000015 05aa25db 00000006 00483c0c hadam3p_afr_7.22_windows_intelx!+0x0 0082fee8 75532def 00000006 70b1d296 75532c30 75532c30 hadam3p_afr_7.22_windows_intelx!+0x0 0082ff80 755b3744 80000006 755b3720 7083dcf6 0082ffdc KERNELBASE!CtrlRoutine+0x0 0082ff94 77de9e54 80000006 720f1fbd 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0082ffdc 77de9e1f ffffffff 77e0d70e 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 0082ffec 00000000 75532c30 80000006 00000000 00905a4d ntdll!RtlSetCurrentTransaction+0x0 * Dump of thread ID 664 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=0019f98c ecx=00000000 edx=00000000 esi=00000000 edi=000000c5 eip=77df8e4c esp=0019f94c ebp=0019f9b0 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0019f9b0 754d14bf 000000c8 00000000 0019f9d4 00467c01 ntdll!NtDelayExecution+0x0 0019f9c0 00467c01 000000c8 000000c8 00000000 0019fa20 KERNELBASE!Sleep+0x0 0019f9d4 004024be a0000000 027f9999 00000000 004cf850 hadam3p_afr_7.22_windows_intelx!+0x0 0019fa20 00402977 00000001 004cf838 004cfb70 00000000 hadam3p_afr_7.22_windows_intelx!+0x0 0019fa54 00403b4e 00000000 05312673 0048549e 00000000 hadam3p_afr_7.22_windows_intelx!+0x0 0019fd40 00403d60 0019fef0 00403e13 004d0290 0019fd60 hadam3p_afr_7.22_windows_intelx!+0x0 0019fd48 00403e13 004d0290 0019fd60 002b2330 0019fd60 hadam3p_afr_7.22_windows_intelx!+0x0 0019fef0 0048544b 00400000 00000000 002b2375 0000000a hadam3p_afr_7.22_windows_intelx!+0x0 0019ff80 755b3744 7ffde000 755b3720 7018dcf6 0019ffdc hadam3p_afr_7.22_windows_intelx!+0x0 0019ff94 77de9e54 7ffde000 72941fbd 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0019ffdc 77de9e1f ffffffff 77e0d70e 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 0019ffec 00000000 0048549e 7ffde000 00000000 78746341 ntdll!RtlSetCurrentTransaction+0x0 * Dump of thread ID 14420 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=00970000 ecx=00000000 edx=00000000 esi=00970064 edi=00970064 eip=001d039a esp=00acff50 ebp=00acff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 00acff80 755b3744 00970000 755b3720 70addcf6 00acffdc !+0x0 SymFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '001d039a' 00acff94 77de9e54 00970000 72211fbd 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 SymFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '755b3744' 00acffdc 77de9e1f ffffffff 77e0d70e 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 SymFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '77de9e54' 00acffec 00000000 00970064 00970000 00000000 001e1900 ntdll!RtlSetCurrentTransaction+0x0 SymFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '77de9e1f' * Dump of thread ID 14008 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=0123fd1c ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=77df8e4c esp=0123fcdc ebp=0123fd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0123fd40 754d14bf 00000064 00000000 0123ff80 00478504 ntdll!NtDelayExecution+0x0 0123fd50 00478504 00000064 004784e0 004784e0 00000000 KERNELBASE!Sleep+0x0 0123ff80 755b3744 00000000 755b3720 7122dcf6 0123ffdc hadam3p_afr_7.22_windows_intelx!+0x0 0123ff94 77de9e54 00000000 73ae1fbd 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0123ffdc 77de9e1f ffffffff 77e0d70e 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 0123ffec 00000000 004784e0 00000000 00000000 03894000 ntdll!RtlSetCurrentTransaction+0x0 * Dump of thread ID 20128 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=002b80d8 ecx=00000000 edx=00000000 esi=002b7f18 edi=002b2b28 eip=77dfa66c esp=006efdcc ebp=006eff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 006eff80 755b3744 002b2b28 755b3720 706fdcf6 006effdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 006eff94 77de9e54 002b2b28 72e31fbd 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 006effdc 77de9e1f ffffffff 77e0d70e 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 006effec 00000000 77dc4630 002b2b28 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:38:18 (5320): start_timer_thread(): CreateThread() failed, errno 0 21:38:20 (304): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:37:24 (576): start_timer_thread(): CreateThread() failed, errno 0 16:37:26 (4188): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:53:12 (11448): start_timer_thread(): CreateThread() failed, errno 0 18:53:14 (5300): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:04:01 (12804): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:00:38 (1068): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 09:37:16 (11276): start_timer_thread(): CreateThread() failed, errno 0 09:37:18 (6448): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11276, selfPID=6448, iMonCtr=1 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11276, selfPID=11276, iMonCtr=1 20:33:25 (5552): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:51:35 (6676): start_timer_thread(): CreateThread() failed, errno 0 06:51:37 (6744): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 12:24:27 (6984): start_timer_thread(): CreateThread() failed, errno 0 12:24:28 (7060): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 14:29:38 (1828): start_timer_thread(): CreateThread() failed, errno 0 14:29:39 (4400): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:01:06 (2220): start_timer_thread(): CreateThread() failed, errno 0 19:01:08 (6160): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:26:00 (10068): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:12:24 (2152): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:09:19 (12824): start_timer_thread(): CreateThread() failed, errno 0 19:09:20 (12492): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:32:57 (2896): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:31:14 (6904): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 08:57:44 (1336): start_timer_thread(): CreateThread() failed, errno 0 08:57:47 (5844): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5928, selfPID=5928, iMonCtr=1 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Jul 2016 15:03:29	1375535	19407369	hadam3p_afr50_eyql_201412_12_371_010406555_0	34,859	257,546	7.3882
30 Jun 2016 16:03:51	1375535	19407369	hadam3p_afr50_eyql_201412_12_371_010406555_0	23,339	172,486	7.3905
06 Jun 2016 21:48:58	1375535	19407369	hadam3p_afr50_eyql_201412_12_371_010406555_0	11,819	85,805	7.2599