Task 19040956

Name	hadam3p_eu_f5fz_1999_1_010198947_2
Workunit	10138368
Created	28 Oct 2015, 4:50:05 UTC
Sent	28 Oct 2015, 4:50:10 UTC
Report deadline	9 Oct 2016, 10:10:10 UTC
Received	30 Oct 2015, 7:56:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1354724
Run time	20 hours 23 min 30 sec
CPU time	18 hours 40 min 20 sec
Validate state	Invalid
Credit	402.61
Device peak FLOPS	3.32 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v7.26 windows_intelx86
Stderr	<core_client_version>7.6.9</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18724, selfPID=18724, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18752, selfPID=18752, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18792, selfPID=18792, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=17800, selfPID=17800, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4200, selfPID=4200, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=23380, selfPID=23380, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=34912, selfPID=34912, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=35172, selfPID=35172, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=19616, selfPID=19616, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=33936, selfPID=33936, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18316, selfPID=18316, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4772, selfPID=4772, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76CF0192 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0124A4C0 read attempt to address 0x00000044 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76CF0192 Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.5.0 Dump Timestamp : 10/30/15 07:14:02 Install Directory : Data Directory : D:\ProgramData\BOINC Project Symstore : LoadLibraryA( D:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( D:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( D:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( D:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\ProgramData\BOINC\projects\climateprediction.net\hadam3p_eu_f5fz_1999_1_010198947;D:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000001180000 00000000008c2000 D:\ProgramData\BOINC\projects\climateprediction.net\hadrm3p_eu_um_7.26_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : F:\cpdn\cpdn\cpdnboinc\monitor\win_build\Release\hadrm3p.pdb ModLoad: 0000000077200000 0000000000157000 C:\windows\SYSTEM32\ntdll.dll (6.2.9200.17438) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 0000000074ea0000 0000000000130000 C:\windows\SYSTEM32\KERNEL32.DLL (6.2.9200.16859) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 0000000076ce0000 00000000000a7000 C:\windows\SYSTEM32\KERNELBASE.dll (6.2.9200.17366) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 00000000749b0000 0000000000116000 C:\windows\SYSTEM32\USER32.dll (6.2.9200.17044) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 0000000076c30000 00000000000ae000 C:\windows\SYSTEM32\ADVAPI32.dll (6.2.9200.16384) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 0000000075220000 0000000000012000 C:\windows\SYSTEM32\imagehlp.dll (6.2.9200.16745) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.2.9200.16745 (win8_gdr.131018-1503) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16745 ModLoad: 0000000075600000 00000000000fd000 C:\windows\SYSTEM32\GDI32.dll (6.2.9200.17430) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.2.9200.17430 (win8_gdr.150703-0605) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.17430 ModLoad: 0000000076d90000 00000000000b1000 C:\windows\SYSTEM32\msvcrt.dll (7.0.9200.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9200.16384 ModLoad: 0000000074970000 0000000000034000 C:\windows\SYSTEM32\sechost.dll (6.2.9200.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 0000000074820000 00000000000ac000 C:\windows\SYSTEM32\RPCRT4.dll (6.2.9200.17422) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 00000000747f0000 000000000001d000 C:\windows\SYSTEM32\SspiCli.dll (6.2.9200.16864) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.2.9200.16864 (win8_gdr.140309-1509) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16864 ModLoad: 00000000747e0000 0000000000009000 C:\windows\SYSTEM32\CRYPTBASE.dll (6.2.9200.16384) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 0000000074780000 0000000000051000 C:\windows\SYSTEM32\bcryptPrimitives.dll (6.2.9200.16384) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 00000000741f0000 0000000000129000 C:\windows\SYSTEM32\dbghelp.dll (6.2.9200.16384) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 ModLoad: 0000000073e50000 0000000000008000 C:\windows\SYSTEM32\version.dll (6.2.9200.16384) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.2.9200.16384 (win8_rtm.120725-1247) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.2.9200.16384 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 4, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 53, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 106096, QuotaPeakPagedPoolUsage: 106168 QuotaNonPagedPoolUsage: 6912, QuotaPeakNonPagedPoolUsage: 6912 - Virtual Memory Usage - VirtualSize: 940675072, PeakVirtualSize: 948908032 - Pagefile Usage - PagefileUsage: 16556032, PeakPagefileUsage: 24776704 - Working Set Size - WorkingSetSize: 7938048, PeakWorkingSetSize: 7938048, PageFaultCount: 1984 * Dump of thread ID 61640 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 312002.000000, Wait Time: 20588732.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x76CF0192 - Registers - eax=1316f790 ebx=7ebeb000 ecx=00000003 edx=00000000 esi=014e9090 edi=015e0ac0 eip=76cf0192 esp=1316f790 ebp=1316f7e8 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1316f7e8 01490947 e06d7363 00000001 00000003 1316f814 KERNELBASE!RaiseException+0x0 1316f820 0148daf2 1316f830 01505bf4 014e9090 014fdcec hadrm3p_eu_um_7.26_windows_inte!+0x0 1316f840 01188272 01000000 01187b59 13490000 13493b26 hadrm3p_eu_um_7.26_windows_inte!+0x0 1316fc6c 0148f91d 01180000 00000000 00f258a6 0000000a hadrm3p_eu_um_7.26_windows_inte!+0x0 1316fcfc 74ec86e3 7ebeb000 1316fd4c 7725bd79 7ebeb000 hadrm3p_eu_um_7.26_windows_inte!+0x0 1316fd08 7725bd79 7ebeb000 01d49ad6 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1316fd4c 7725bd4c 0148f970 7ebeb000 ffffffff 772d21af ntdll!RtlInitializeExceptionChain+0x0 1316fd64 00000000 0148f970 7ebeb000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 82568 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 20588732.000000 - Registers - eax=014126f0 ebx=37aefb28 ecx=00000000 edx=00000000 esi=37aefb28 edi=00000000 eip=7722dd1c esp=37aefae8 ebp=37aefb4c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 37aefb4c 76ce119c 00000064 00000000 37aefd8c 01412714 ntdll!NtDelayExecution+0x0 37aefb5c 01412714 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 37aefd8c 74ec86e3 00000000 37aefddc 7725bd79 00000000 hadrm3p_eu_um_7.26_windows_inte!+0x0 37aefd98 7725bd79 00000000 256c9a46 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 37aefddc 7725bd4c 014126f0 00000000 ffffffff 772d21af ntdll!RtlInitializeExceptionChain+0x0 37aefdf4 00000000 014126f0 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4284, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 07:14:22 (4284): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_f5fz_1999_1_010198947_2_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2015 14:16:23	1354724	19040956	hadam3p_eu_f5fz_1999_1_010198947_2	23,339	48,490	2.0776
05 Nov 2015 14:10:45	1354724	19040956	hadam3p_eu_f5fz_1999_1_010198947_2	11,819	24,888	2.1058