Task 16047974

Name	hadcm3n_oeqq_1900_40_008474229_1
Workunit	8625068
Created	28 Sep 2013, 8:24:20 UTC
Sent	28 Sep 2013, 9:05:26 UTC
Report deadline	28 Dec 2013, 16:32:37 UTC
Received	12 Oct 2013, 23:52:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1284993
Run time	7 days 17 hours 26 min 56 sec
CPU time	6 days 7 hours 53 min 48 sec
Validate state	Invalid
Credit	5,598.72
Device peak FLOPS	2.43 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 70a60000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 71fc0000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6fca0000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 72270000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7601.18177) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.1.7601.18177 (win7sp1_gdr.130605-1534) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18177 ModLoad: 75340000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75b00000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : cfgmgr32.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 Get Product Name Failed. ModLoad: 75ef0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.1.7601.17676 Company Name : Microsoft Corporation Product Name : Product Version : 6.1.7601.17676 ModLoad: 756f0000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75060000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : devobj.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 ModLoad: 742f0000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 74e20000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.1.7601.17787 (win7sp1_gdr.120229-1502) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17787 ModLoad: 75690000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 75e20000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6cf30000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 73540000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 2496 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x0072B85F - Registers - eax=00000001 ebx=00000001 ecx=007bb5ee edx=001d0174 esi=001dfeb8 edi=00000000 eip=0072b85f esp=102b0a5c ebp=102b0d54 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 102b0d54 0040259b 102b0d64 132d00dc 425c3a63 434e494f hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ddc 00732bb0 ffffffff 102b0e18 102b0e20 102b0e10 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e30 00748ef6 ffffffff 102b0e48 00000000 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e58 00408b3e 132d0000 132d3b0e f60fcb96 00408b46 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e68 00408b46 00000018 10333148 10333154 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ec8 004d9dd4 10333154 004d9dd4 1033311c 103a01f0 hadcm3n_um_6.07_windows_intelx8!+0x0 10333148 007055a7 00822310 1132f798 007055a7 1132f89c hadcm3n_um_6.07_windows_intelx8!+0x0 10333154 007055a7 1132f89c 1132f8a0 1132f8a4 1132f8a8 hadcm3n_um_6.07_windows_intelx8!+0x0 10333158 1132f89c 1132f8a0 1132f8a4 1132f8a8 39a80020 hadcm3n_um_6.07_windows_intelx8!+0x0 1033315c 1132f8a0 1132f8a4 1132f8a8 39a80020 39a86620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f89c' 10333160 1132f8a4 1132f8a8 39a80020 39a86620 39a86620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a0' 10333164 1132f8a8 39a80020 39a86620 39a86620 39a86620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a4' 10333168 39a80020 39a86620 39a86620 39a86620 103671a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a8' 1033316c 39a86620 39a86620 39a86620 103671a0 103675a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a80020' 10333170 39a86620 39a86620 103671a0 103675a0 10367614 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a86620' 10333174 39a86620 103671a0 103675a0 10367614 10367618 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a86620' 10333178 103671a0 103675a0 10367614 10367618 1036761c hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a86620' 1033317c 103675a0 10367614 10367618 1036761c 10367620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103671a0' 103671a0 00000000 00000001 00000000 00000001 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103675a0' * Dump of thread ID 3236 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=33f4ff48 edi=00000000 eip=7727fd91 esp=33f4ff04 ebp=33f4ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 33f4ff6c 758e4498 00000064 00000000 33f4ff88 00749924 ntdll!ZwDelayExecution+0x0 33f4ff7c 00749924 00000064 33f4ff94 74e6336a 00000000 KERNELBASE!Sleep+0x0 33f4ff88 74e6336a 00000000 33f4ffd4 77299f72 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 33f4ff94 77299f72 00000000 4b57d277 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 33f4ffd4 77299f45 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 33f4ffec 00000000 00749910 00000000 00000000 34460000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x0072B85F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/12/13 22:00:27 Install Directory : Data Directory : c:\BOINC\ProgramData Project Symstore : LoadLibraryA( c:\BOINC\ProgramData\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( c:\BOINC\ProgramData\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( c:\BOINC\ProgramData\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( c:\BOINC\ProgramData\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: c:\BOINC\ProgramData\projects\climateprediction.net\hadcm3n_oeqq_1900_40_008474229;c:\BOINC\ProgramData\projects\climateprediction.net ModLoad: 00400000 00b28000 c:\BOINC\ProgramData\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77260000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.18247) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 739b0000 0003f000 C:\Program Files\AVAST Software\Avast\snxhk.dll (8.0.1497.376) (-exported- Symbols Loaded) Linked PDB Filename : d:\Avast800\src\BUILDS\Release\x86\snxhk.pdb File Version : 8.0.1497.376 Company Name : AVAST Software Product Name : avast! Antivirus Product Version : 8.0.1497.376 ModLoad: 74e50000 00110000 C:\Windows\syswow64\KERNEL32.dll (6.1.7601.18229) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 758d0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.18229) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 750e0000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 76dd0000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 75a60000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.18177) (-exported- Symbols Loaded) Linked PDB Filename : wlpk.pdb File Version : 6.1.7601.18177 (win7sp1_gdr.130605-1534) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18177 ModLoad: 754e0000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.18009) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.7601.18009 (win7sp1_gdr.121121-1431) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.7601.18009 ModLoad: 75b30000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.7601.17744 (win7sp1_gdr.111215-1535) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.7601.17744 ModLoad: 74f60000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.18247) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75080000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 751e0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.18205) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 74dc0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17940) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.1.7601.17940 (win7sp1_gdr.120824-0334) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17940 ModLoad: 74db0000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 70a60000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 71fc0000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6fca0000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 72270000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7601.18177) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.1.7601.18177 (win7sp1_gdr.130605-1534) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18177 ModLoad: 75340000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75b00000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : cfgmgr32.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 Get Product Name Failed. ModLoad: 75ef0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.1.7601.17676 Company Name : Microsoft Corporation Product Name : Product Version : 6.1.7601.17676 ModLoad: 756f0000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75060000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : devobj.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 ModLoad: 742f0000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 74e20000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.1.7601.17787 (win7sp1_gdr.120229-1502) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17787 ModLoad: 75690000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 75e20000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6ce40000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 73540000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 3084 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x0072B85F - Registers - eax=00000001 ebx=00000001 ecx=007bb5ee edx=131b0174 esi=131bfeb8 edi=00000000 eip=0072b85f esp=102b0a5c ebp=102b0d54 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 102b0d54 0040259b 102b0d64 131c00dc 425c3a63 434e494f hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ddc 00732bb0 ffffffff 102b0e18 102b0e20 102b0e10 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e30 00748ef6 ffffffff 102b0e48 00000000 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e58 00408b3e 131c0000 131c3b0e 846419ee 00408b46 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e68 00408b46 00000018 10333148 10333154 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ec8 004d9dd4 10333154 004d9dd4 1033311c 103a01f0 hadcm3n_um_6.07_windows_intelx8!+0x0 10333148 007055a7 00822310 1132f798 007055a7 1132f89c hadcm3n_um_6.07_windows_intelx8!+0x0 10333154 007055a7 1132f89c 1132f8a0 1132f8a4 1132f8a8 hadcm3n_um_6.07_windows_intelx8!+0x0 10333158 1132f89c 1132f8a0 1132f8a4 1132f8a8 39970020 hadcm3n_um_6.07_windows_intelx8!+0x0 1033315c 1132f8a0 1132f8a4 1132f8a8 39970020 39976620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f89c' 10333160 1132f8a4 1132f8a8 39970020 39976620 39976620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a0' 10333164 1132f8a8 39970020 39976620 39976620 39976620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a4' 10333168 39970020 39976620 39976620 39976620 103671a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a8' 1033316c 39976620 39976620 39976620 103671a0 103675a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39970020' 10333170 39976620 39976620 103671a0 103675a0 10367614 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39976620' 10333174 39976620 103671a0 103675a0 10367614 10367618 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39976620' 10333178 103671a0 103675a0 10367614 10367618 1036761c hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39976620' 1033317c 103675a0 10367614 10367618 1036761c 10367620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103671a0' 103671a0 00000000 00000001 00000000 00000001 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103675a0' * Dump of thread ID 1648 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=33e3ff48 edi=00000000 eip=7727fd91 esp=33e3ff04 ebp=33e3ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 33e3ff6c 758e4498 00000064 00000000 33e3ff88 00749924 ntdll!ZwDelayExecution+0x0 33e3ff7c 00749924 00000064 33e3ff94 74e6336a 00000000 KERNELBASE!Sleep+0x0 33e3ff88 74e6336a 00000000 33e3ffd4 77299f72 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 33e3ff94 77299f72 00000000 4b437d56 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 33e3ffd4 77299f45 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 33e3ffec 00000000 00749910 00000000 00000000 34350000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x0072B85F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/12/13 22:02:10 Install Directory : Data Directory : c:\BOINC\ProgramData Project Symstore : LoadLibraryA( c:\BOINC\ProgramData\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( c:\BOINC\ProgramData\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( c:\BOINC\ProgramData\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( c:\BOINC\ProgramData\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: c:\BOINC\ProgramData\projects\climateprediction.net\hadcm3n_oeqq_1900_40_008474229;c:\BOINC\ProgramData\projects\climateprediction.net ModLoad: 00400000 00b28000 c:\BOINC\ProgramData\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77260000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.18247) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 739b0000 0003f000 C:\Program Files\AVAST Software\Avast\snxhk.dll (8.0.1497.376) (-exported- Symbols Loaded) Linked PDB Filename : d:\Avast800\src\BUILDS\Release\x86\snxhk.pdb File Version : 8.0.1497.376 Company Name : AVAST Software Product Name : avast! Antivirus Product Version : 8.0.1497.376 ModLoad: 74e50000 00110000 C:\Windows\syswow64\KERNEL32.dll (6.1.7601.18229) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 758d0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.18229) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 750e0000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 76dd0000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 75a60000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.18177) (-exported- Symbols Loaded) Linked PDB Filename : wlpk.pdb File Version : 6.1.7601.18177 (win7sp1_gdr.130605-1534) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18177 ModLoad: 754e0000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.18009) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.7601.18009 (win7sp1_gdr.121121-1431) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.7601.18009 ModLoad: 75b30000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.7601.17744 (win7sp1_gdr.111215-1535) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.7601.17744 ModLoad: 74f60000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.18247) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75080000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 751e0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.18205) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 74dc0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17940) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.1.7601.17940 (win7sp1_gdr.120824-0334) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17940 ModLoad: 74db0000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 70a60000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 71fc0000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6fca0000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 72270000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7601.18177) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.1.7601.18177 (win7sp1_gdr.130605-1534) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18177 ModLoad: 75340000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75b00000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : cfgmgr32.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 Get Product Name Failed. ModLoad: 75ef0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.1.7601.17676 Company Name : Microsoft Corporation Product Name : Product Version : 6.1.7601.17676 ModLoad: 756f0000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75060000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : devobj.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 ModLoad: 742f0000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 74e20000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.1.7601.17787 (win7sp1_gdr.120229-1502) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17787 ModLoad: 75690000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 75e20000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6cf30000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 73540000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 4988 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x0072B85F - Registers - eax=00000001 ebx=00000001 ecx=007bb5ee edx=13040174 esi=1304feb8 edi=00000000 eip=0072b85f esp=102b0a5c ebp=102b0d54 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 102b0d54 0040259b 102b0d64 131100dc 425c3a63 434e494f hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ddc 00732bb0 ffffffff 102b0e18 102b0e20 102b0e10 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e30 00748ef6 ffffffff 102b0e48 00000000 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e58 00408b3e 13110000 13113b0e 52a455c9 00408b46 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e68 00408b46 00000018 10333148 10333154 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ec8 004d9dd4 10333154 004d9dd4 1033311c 103a01f0 hadcm3n_um_6.07_windows_intelx8!+0x0 10333148 007055a7 00822310 1132f798 007055a7 1132f89c hadcm3n_um_6.07_windows_intelx8!+0x0 10333154 007055a7 1132f89c 1132f8a0 1132f8a4 1132f8a8 hadcm3n_um_6.07_windows_intelx8!+0x0 10333158 1132f89c 1132f8a0 1132f8a4 1132f8a8 398c0020 hadcm3n_um_6.07_windows_intelx8!+0x0 1033315c 1132f8a0 1132f8a4 1132f8a8 398c0020 398c6620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f89c' 10333160 1132f8a4 1132f8a8 398c0020 398c6620 398c6620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a0' 10333164 1132f8a8 398c0020 398c6620 398c6620 398c6620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a4' 10333168 398c0020 398c6620 398c6620 398c6620 103671a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a8' 1033316c 398c6620 398c6620 398c6620 103671a0 103675a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '398c0020' 10333170 398c6620 398c6620 103671a0 103675a0 10367614 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '398c6620' 10333174 398c6620 103671a0 103675a0 10367614 10367618 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '398c6620' 10333178 103671a0 103675a0 10367614 10367618 1036761c hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '398c6620' 1033317c 103675a0 10367614 10367618 1036761c 10367620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103671a0' 103671a0 00000000 00000001 00000000 00000001 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103675a0' * Dump of thread ID 5084 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=33d8ff48 edi=00000000 eip=7727fd91 esp=33d8ff04 ebp=33d8ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 33d8ff6c 758e4498 00000064 00000000 33d8ff88 00749924 ntdll!ZwDelayExecution+0x0 33d8ff7c 00749924 00000064 33d8ff94 74e6336a 00000000 KERNELBASE!Sleep+0x0 33d8ff88 74e6336a 00000000 33d8ffd4 77299f72 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 33d8ff94 77299f72 00000000 4b76e471 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 33d8ffd4 77299f45 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 33d8ffec 00000000 00749910 00000000 00000000 342a0000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x0072B85F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/12/13 22:03:54 Install Directory : Data Directory : c:\BOINC\ProgramData Project Symstore : LoadLibraryA( c:\BOINC\ProgramData\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( c:\BOINC\ProgramData\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( c:\BOINC\ProgramData\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( c:\BOINC\ProgramData\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: c:\BOINC\ProgramData\projects\climateprediction.net\hadcm3n_oeqq_1900_40_008474229;c:\BOINC\ProgramData\projects\climateprediction.net ModLoad: 00400000 00b28000 c:\BOINC\ProgramData\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77260000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.18247) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 739b0000 0003f000 C:\Program Files\AVAST Software\Avast\snxhk.dll (8.0.1497.376) (-exported- Symbols Loaded) Linked PDB Filename : d:\Avast800\src\BUILDS\Release\x86\snxhk.pdb File Version : 8.0.1497.376 Company Name : AVAST Software Product Name : avast! Antivirus Product Version : 8.0.1497.376 ModLoad: 74e50000 00110000 C:\Windows\syswow64\KERNEL32.dll (6.1.7601.18229) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 758d0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.18229) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7601.18015 (win7sp1_gdr.121129-1432) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18015 ModLoad: 750e0000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 76dd0000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 75a60000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.18177) (-exported- Symbols Loaded) Linked PDB Filename : wlpk.pdb File Version : 6.1.7601.18177 (win7sp1_gdr.130605-1534) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18177 ModLoad: 754e0000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.18009) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.7601.18009 (win7sp1_gdr.121121-1431) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.7601.18009 ModLoad: 75b30000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.7601.17744 (win7sp1_gdr.111215-1535) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.7601.17744 ModLoad: 74f60000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.18247) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75080000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 751e0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.18205) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 74dc0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17940) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.1.7601.17940 (win7sp1_gdr.120824-0334) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17940 ModLoad: 74db0000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 70a60000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 71fc0000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6fca0000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 72270000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7601.18177) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.1.7601.18177 (win7sp1_gdr.130605-1534) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.18177 ModLoad: 75340000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75b00000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : cfgmgr32.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 Get Product Name Failed. ModLoad: 75ef0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.1.7601.17676 Company Name : Microsoft Corporation Product Name : Product Version : 6.1.7601.17676 ModLoad: 756f0000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75060000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded) Linked PDB Filename : devobj.pdb File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17621 ModLoad: 742f0000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 74e20000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.1.7601.17787 (win7sp1_gdr.120229-1502) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17787 ModLoad: 75690000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 75e20000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 6ce40000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7601.17514 ModLoad: 73540000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 4072 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Breakpoint Encountered (0x80000003) at address 0x0072B85F - Registers - eax=00000001 ebx=00000001 ecx=007bb5ee edx=13240174 esi=1324feb8 edi=00000000 eip=0072b85f esp=102b0a5c ebp=102b0d54 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 102b0d54 0040259b 102b0d64 132b00dc 425c3a63 434e494f hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ddc 00732bb0 ffffffff 102b0e18 102b0e20 102b0e10 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e30 00748ef6 ffffffff 102b0e48 00000000 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e58 00408b3e 132b0000 132b3b0e e088969a 00408b46 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0e68 00408b46 00000018 10333148 10333154 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 102b0ec8 004d9dd4 10333154 004d9dd4 1033311c 103a01f0 hadcm3n_um_6.07_windows_intelx8!+0x0 10333148 007055a7 00822310 1132f798 007055a7 1132f89c hadcm3n_um_6.07_windows_intelx8!+0x0 10333154 007055a7 1132f89c 1132f8a0 1132f8a4 1132f8a8 hadcm3n_um_6.07_windows_intelx8!+0x0 10333158 1132f89c 1132f8a0 1132f8a4 1132f8a8 39a60020 hadcm3n_um_6.07_windows_intelx8!+0x0 1033315c 1132f8a0 1132f8a4 1132f8a8 39a60020 39a66620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f89c' 10333160 1132f8a4 1132f8a8 39a60020 39a66620 39a66620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a0' 10333164 1132f8a8 39a60020 39a66620 39a66620 39a66620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a4' 10333168 39a60020 39a66620 39a66620 39a66620 103671a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '1132f8a8' 1033316c 39a66620 39a66620 39a66620 103671a0 103675a0 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a60020' 10333170 39a66620 39a66620 103671a0 103675a0 10367614 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a66620' 10333174 39a66620 103671a0 103675a0 10367614 10367618 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a66620' 10333178 103671a0 103675a0 10367614 10367618 1036761c hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '39a66620' 1033317c 103675a0 10367614 10367618 1036761c 10367620 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103671a0' 103671a0 00000000 00000001 00000000 00000001 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 SymFromAddr(): GetLastError = '126' SymGetLineFromAddr(): GetLastError = '126' SymGetModuleInfo(): GetLastError = '126' Address = '103675a0' * Dump of thread ID 3876 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=33f2ff48 edi=00000000 eip=7727fd91 esp=33f2ff04 ebp=33f2ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 33f2ff6c 758e4498 00000064 00000000 33f2ff88 00749924 ntdll!ZwDelayExecution+0x0 33f2ff7c 00749924 00000064 33

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Oct 2013 18:23:08	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	466,560	658,916	1.4123
12 Oct 2013 08:04:29	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	440,640	621,891	1.4113
11 Oct 2013 21:03:52	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	414,720	584,228	1.4087
10 Oct 2013 02:17:32	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	388,800	546,757	1.4063
08 Oct 2013 23:48:30	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	362,880	509,491	1.4040
07 Oct 2013 21:31:55	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	336,960	472,571	1.4025
06 Oct 2013 18:50:25	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	311,040	434,597	1.3972
06 Oct 2013 08:04:32	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	285,120	396,001	1.3889
05 Oct 2013 21:23:55	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	259,200	357,660	1.3799
05 Oct 2013 10:59:42	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	233,280	320,418	1.3735
05 Oct 2013 00:50:27	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	207,360	283,967	1.3694
03 Oct 2013 22:36:05	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	181,440	247,481	1.3640
02 Oct 2013 04:26:58	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	155,520	211,101	1.3574
01 Oct 2013 02:07:48	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	129,600	174,127	1.3436
29 Sep 2013 23:41:04	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	103,680	136,825	1.3197
29 Sep 2013 13:20:47	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	77,760	99,661	1.2816
29 Sep 2013 03:52:01	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	51,840	66,170	1.2764
28 Sep 2013 18:39:10	1284993	16047974	hadcm3n_oeqq_1900_40_008474229_1	25,920	33,136	1.2784