Task 15601440

Name	hadcm3n_4iqq_1940_40_008305195_1
Workunit	8456330
Created	8 Feb 2013, 19:09:16 UTC
Sent	8 Feb 2013, 19:24:31 UTC
Report deadline	11 May 2013, 2:51:42 UTC
Received	4 Mar 2013, 16:34:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1191046
Run time	10 days 8 hours 7 min 28 sec
CPU time	10 days 6 hours 43 min 5 sec
Validate state	Invalid
Credit	9,642.24
Device peak FLOPS	3.18 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 03/04/13 10:33:07 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_4iqq_1940_40_008305195;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77dc0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7600.16915) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16915 (win7_gdr.111116-1506) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16915 ModLoad: 768c0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 775a0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7600.17206) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 77290000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 10708 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F - Registers - eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=775ac41f esp=1132f9c8 ebp=1132fa18 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!RaiseException+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 11503941 1132faec 115038c9 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 768d3677 7efde000 1132ffd4 77df9f42 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77df9f42 7efde000 38c184bf 00000000 00000000 kernel32!+0x0 1132ffd4 77df9f15 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 1136 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=339dff48 edi=00000000 eip=77ddfd21 esp=339dff04 ebp=339dff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 339dff6c 775b4498 00000064 00000000 339dff88 00749924 ntdll!ZwDelayExecution+0x0 339dff7c 00749924 00000064 339dff94 768d3677 00000000 KERNELBASE!Sleep+0x0 339dff88 768d3677 00000000 339dffd4 77df9f42 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 339dff94 77df9f42 00000000 1a6e84bf 00000000 00000000 kernel32!+0x0 339dffd4 77df9f15 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 339dffec 00000000 00749910 00000000 00000000 002500a0 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8344, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 03/04/13 10:33:10 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_4iqq_1940_40_008305195;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77dc0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7600.16915) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16915 (win7_gdr.111116-1506) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16915 ModLoad: 768c0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 775a0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 77290000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77410000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 774a0000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wlpk.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77920000 0009e000 C:\Windows\syswow64\USP10.dll (1.626.7600.17174) (-nosymbols- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.7600.17174 (win7_gdr.121121-1913) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.7600.17174 ModLoad: 769d0000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7600.16930) (-nosymbols- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.7600.16930 (win7_gdr.111215-1720) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.7600.16930 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 11400 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F - Registers - eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=775ac41f esp=1132f9c8 ebp=1132fa18 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 11453941 1132faec 114538c9 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 768d3677 7efde000 1132ffd4 77df9f42 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77df9f42 7efde000 38c1b963 00000000 00000000 kernel32!+0x0 1132ffd4 77df9f15 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7efde000 00000000 75733c00 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 12456 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=3385ff48 edi=00000000 eip=77ddfd21 esp=3385ff04 ebp=3385ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3385ff6c 775b4498 00000064 00000000 3385ff88 00749924 ntdll!ZwDelayExecution+0x0 3385ff7c 00749924 00000064 3385ff94 768d3677 00000000 KERNELBASE!+0x0 3385ff88 768d3677 00000000 3385ffd4 77df9f42 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3385ff94 77df9f42 00000000 1a76b963 00000000 00000000 kernel32!+0x0 3385ffd4 77df9f15 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3385ffec 00000000 00749910 00000000 00000000 113800a0 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8344, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 03/04/13 10:33:12 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_4iqq_1940_40_008305195;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77dc0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7600.16915) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16915 (win7_gdr.111116-1506) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16915 ModLoad: 768c0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7600.17206) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 775a0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 77290000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77410000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 13888 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F - Registers - eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=775ac41f esp=1132f9c8 ebp=1132fa18 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 114d3941 1132faec 114d38c9 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 768d3677 7efde000 1132ffd4 77df9f42 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77df9f42 7efde000 38c1ae8f 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 1132ffd4 77df9f15 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 15428 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=3395ff48 edi=00000000 eip=77ddfd21 esp=3395ff04 ebp=3395ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3395ff6c 775b4498 00000064 00000000 3395ff88 00749924 ntdll!ZwDelayExecution+0x0 3395ff7c 00749924 00000064 3395ff94 768d3677 00000000 KERNELBASE!+0x0 3395ff88 768d3677 00000000 3395ffd4 77df9f42 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3395ff94 77df9f42 00000000 1a66ae8f 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 3395ffd4 77df9f15 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3395ffec 00000000 00749910 00000000 00000000 002500a0 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8344, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 03/04/13 10:33:15 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_4iqq_1940_40_008305195;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77dc0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7600.16915) (-nosymbols- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16915 (win7_gdr.111116-1506) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16915 ModLoad: 768c0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 775a0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7600.17206) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 77290000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77410000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 774a0000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : wlpk.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77920000 0009e000 C:\Windows\syswow64\USP10.dll (1.626.7600.17174) (-nosymbols- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.7600.17174 (win7_gdr.121121-1913) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.7600.17174 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 3244 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F - Registers - eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=775ac41f esp=1132f9c8 ebp=1132fa18 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!RaiseException+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 114f3941 1132faec 114f38c9 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 768d3677 7efde000 1132ffd4 77df9f42 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77df9f42 7efde000 38c1a438 00000000 00000000 kernel32!+0x0 1132ffd4 77df9f15 007bde7f 7efde000 00000000 00000000 ntdll!+0x0 1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!+0x0 * Dump of thread ID 14092 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=339cff48 edi=00000000 eip=77ddfd21 esp=339cff04 ebp=339cff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 339cff6c 775b4498 00000064 00000000 339cff88 00749924 ntdll!+0x0 339cff7c 00749924 00000064 339cff94 768d3677 00000000 KERNELBASE!Sleep+0x0 339cff88 768d3677 00000000 339cffd4 77df9f42 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 339cff94 77df9f42 00000000 1a6fa438 00000000 00000000 kernel32!+0x0 339cffd4 77df9f15 00749910 00000000 00000000 00000000 ntdll!+0x0 339cffec 00000000 00749910 00000000 00000000 12d400a0 ntdll!+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8344, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 03/04/13 10:33:17 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_4iqq_1940_40_008305195;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77dc0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7600.16915) (-nosymbols- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16915 (win7_gdr.111116-1506) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16915 ModLoad: 768c0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 775a0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7600.17206) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 77290000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77410000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77920000 0009e000 C:\Windows\syswow64\USP10.dll (1.626.7600.17174) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.7600.17174 (win7_gdr.121121-1913) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.7600.17174 ModLoad: 769d0000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7600.16930) (-nosymbols- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.7600.16930 (win7_gdr.111215-1720) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.7600.16930 ModLoad: 77820000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 768a0000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 774b0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 75920000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7600.17035) (-nosymbols- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.1.7600.17035 (win7_gdr.120601-1506) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17035 ModLoad: 75910000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 6576 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F - Registers - eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=775ac41f esp=1132f9c8 ebp=1132fa18 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!RaiseException+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 11463941 1132faec 114638c9 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 768d3677 7efde000 1132ffd4 77df9f42 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77df9f42 7efde000 38c15bba 00000000 00000000 kernel32!+0x0 1132ffd4 77df9f15 007bde7f 7efde000 00000000 00000000 ntdll!+0x0 1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!+0x0 * Dump of thread ID 16984 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=3389ff48 edi=00000000 eip=77ddfd21 esp=3389ff04 ebp=3389ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3389ff6c 775b4498 00000064 00000000 3389ff88 00749924 ntdll!+0x0 3389ff7c 00749924 00000064 3389ff94 768d3677 00000000 KERNELBASE!Sleep+0x0 3389ff88 768d3677 00000000 3389ffd4 77df9f42 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3389ff94 77df9f42 00000000 1a7a5bba 00000000 00000000 kernel32!+0x0 3389ffd4 77df9f15 00749910 00000000 00000000 00000000 ntdll!+0x0 3389ffec 00000000 00749910 00000000 00000000 113600a0 ntdll!+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8344, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 03/04/13 10:33:20 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_4iqq_1940_40_008305195;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77dc0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7600.16915) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.1.7600.16915 (win7_gdr.111116-1506) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16915 ModLoad: 768c0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 775a0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7600.17206) (-nosymbols- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.1.7600.17206 (win7_gdr.130103-1435) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.17206 ModLoad: 77290000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 ModLoad: 77410000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.1.7600.16385 (win7_rtm.090713-1255) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.1.7600.16385 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 * Dump of thread ID 16596 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x775AC41F - Registers - eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=775ac41f esp=1132f9c8 ebp=1132fa18 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 002e3941 1132faec 002e38c9 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 768d3677 7efde000 1132ffd4 77df9f42 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77df9f42 7efde000 38c151db 00000000 00000000 kernel32!+0x0 1132ffd4 77df9f15 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 15040 (state: Initialized): * - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=3393ff48 edi=00000000 eip=77ddfd21 esp=3393ff04 ebp=3393ff6c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3393ff6c 775b4498 00000064 00000000 3393ff88 00749924 ntdll!ZwDelayExecution+0x0 3393ff7c 00749924 00000064 3393ff94 768d3677 00000000 KERNELBASE!+0x0 3393ff88 768d3677 00000000 3393ffd4 77df9f42 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3393ff94 77df9f42 00000000 1a6051db 00000000 00000000 kernel32!+0x0 3393ffd4 77df9f15 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3393ffec 00000000 00749910 00000000 00000000 002800a0 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8344, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Feb 2013 04:38:46	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	803,520	875,350	1.0894
23 Feb 2013 18:54:03	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	777,600	847,173	1.0895
23 Feb 2013 09:02:10	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	751,680	819,026	1.0896
22 Feb 2013 22:59:55	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	725,760	790,490	1.0892
22 Feb 2013 13:10:10	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	699,840	762,335	1.0893
22 Feb 2013 03:15:47	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	673,920	733,894	1.0890
21 Feb 2013 17:26:17	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	648,000	705,802	1.0892
21 Feb 2013 07:24:21	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	622,080	677,403	1.0889
20 Feb 2013 21:36:44	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	596,160	649,291	1.0891
20 Feb 2013 11:48:41	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	570,240	621,136	1.0893
20 Feb 2013 01:51:41	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	544,320	592,674	1.0888
19 Feb 2013 16:01:01	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	518,400	564,564	1.0891
19 Feb 2013 06:07:41	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	492,480	536,184	1.0887
18 Feb 2013 20:23:40	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	466,560	508,146	1.0891
18 Feb 2013 10:33:42	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	440,640	480,041	1.0894
18 Feb 2013 00:46:27	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	414,720	451,685	1.0891
17 Feb 2013 14:45:28	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	388,800	423,266	1.0886
17 Feb 2013 04:31:08	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	362,880	394,097	1.0860
16 Feb 2013 16:55:58	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	336,960	365,246	1.0839
15 Feb 2013 20:24:46	1191046	15601440	hadcm3n_4iqq_1940_40_008305195_1	311,040	336,133	1.0807