Task 16194421

Name	hadcm3n_od6v_1900_40_008472218_3
Workunit	8623057
Created	1 Jan 2014, 18:12:37 UTC
Sent	1 Jan 2014, 18:12:55 UTC
Report deadline	3 Apr 2014, 1:40:06 UTC
Received	18 Jan 2014, 10:12:51 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1259513
Run time	15 days 11 hours 36 min 27 sec
CPU time	15 days 2 hours 43 min
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 01/09/14 14:45:04 Install Directory : Data Directory : D:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_od6v_1900_40_008472218;D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 111813, Write: 0, Other 244517 - I/O Transfers Counters - Read: 0, Write: 8241, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124432, QuotaPeakPagedPoolUsage: 124432 QuotaNonPagedPoolUsage: 5664, QuotaPeakNonPagedPoolUsage: 5712 - Virtual Memory Usage - VirtualSize: 1051312128, PeakVirtualSize: 1051312128 - Pagefile Usage - PagefileUsage: 231915520, PeakPagefileUsage: 231915520 - Working Set Size - WorkingSetSize: 226951168, PeakWorkingSetSize: 226951168, PageFaultCount: 59586 * Dump of thread ID 3064 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 139375008.000000, User Time: 85830623232.000000, Wait Time: 571347.000000 - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 - Registers - eax=00000007 ebx=0f270908 ecx=00000000 edx=10473da4 esi=0f52baa0 edi=0f253c20 eip=006c6d27 esp=0f104900 ebp=0f2708f8 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0f2708f8 0064a92b 0f251ae0 00000488 0f2e2dcc 0064a92b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006c6d27' 0f2e2db8 00455431 0f52b618 00000122 0f525b78 0f4e2690 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0064a92b' 0f4b915c 0044f0de 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00455431' 0f6d5e78 0065501d 40aaa4e2 40b06e48 100cddf8 0065501d !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0044f0de' 100cdde8 005df71b 00000002 00000002 10332864 005df71b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0065501d' 10332858 005da7e1 10332d44 3de20020 005da7e1 1132f8dc !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005df71b' 10332d84 005dc121 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005da7e1' 10333188 007061a7 1031e8d0 10317b50 1132f7d8 007061a7 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005dc121' 1132f7c8 006f3667 1132f8d8 1132f8fc 1132fab8 006f3667 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007061a7' 1132faa8 004083f3 04600000 01000000 00000001 004083f3 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006f3667' 1132fb10 00408130 1132fb24 001324ef 1132fb24 00132451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004083f3' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 80000004 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 1072 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 156250.000000, User Time: 312500.000000, Wait Time: 571343.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=3404ff74 eip=7d61ccc6 esp=3404ff38 ebp=3404ff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3404ff9c 7d4d14ef 00000064 00000000 3404ffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 3404ffac 00749924 00000064 3404ffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 3404ffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 3404ffec 00000000 00749910 00000000 00000000 34560000 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=996, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 01/09/14 14:52:38 Install Directory : Data Directory : D:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_od6v_1900_40_008472218;D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 7060, Write: 0, Other 20433 - I/O Transfers Counters - Read: 0, Write: 747, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124432, QuotaPeakPagedPoolUsage: 124432 QuotaNonPagedPoolUsage: 5584, QuotaPeakNonPagedPoolUsage: 5632 - Virtual Memory Usage - VirtualSize: 1051308032, PeakVirtualSize: 1051308032 - Pagefile Usage - PagefileUsage: 231620608, PeakPagefileUsage: 231620608 - Working Set Size - WorkingSetSize: 226652160, PeakWorkingSetSize: 226652160, PageFaultCount: 55563 * Dump of thread ID 3980 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 10156250.000000, User Time: 4447343616.000000, Wait Time: 600359.000000 - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 - Registers - eax=00000007 ebx=0f270908 ecx=00000000 edx=10473da4 esi=0f52baa0 edi=0f253c20 eip=006c6d27 esp=0f104900 ebp=0f2708f8 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0f2708f8 0064a92b 0f251ae0 00000488 0f2e2dcc 0064a92b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006c6d27' 0f2e2db8 00455431 0f52b618 00000122 0f525b78 0f4e2690 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0064a92b' 0f4b915c 0044f0de 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00455431' 0f6d5e78 0065501d 40ac0813 40afe8c5 100cddf8 0065501d !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0044f0de' 100cdde8 005df71b 00000002 00000002 10332864 005df71b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0065501d' 10332858 005da7e1 10332d44 3de20020 005da7e1 1132f8dc !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005df71b' 10332d84 005dc121 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005da7e1' 10333188 007061a7 1031e8d0 10317b50 1132f7d8 007061a7 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005dc121' 1132f7c8 006f3667 1132f8d8 1132f8fc 1132fab8 006f3667 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007061a7' 1132faa8 004083f3 04600000 01000000 00000001 004083f3 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006f3667' 1132fb10 00408130 1132fb24 001324ef 1132fb24 00132451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004083f3' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 80000004 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 3180 (state: Ready): * - Information - Status: Base Priority: Unknown, Priority: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 600318.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=3404ff74 eip=7d61ccc6 esp=3404ff38 ebp=3404ff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3404ff9c 7d4d14ef 00000064 00000000 3404ffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 3404ffac 00749924 00000064 3404ffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 3404ffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 3404ffec 00000000 00749910 00000000 00000000 34560000 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=996, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 01/09/14 16:39:10 Install Directory : Data Directory : D:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_od6v_1900_40_008472218;D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 78645, Write: 0, Other 173063 - I/O Transfers Counters - Read: 0, Write: 5787, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124432, QuotaPeakPagedPoolUsage: 124432 QuotaNonPagedPoolUsage: 5664, QuotaPeakNonPagedPoolUsage: 5712 - Virtual Memory Usage - VirtualSize: 1051312128, PeakVirtualSize: 1051312128 - Pagefile Usage - PagefileUsage: 231649280, PeakPagefileUsage: 231649280 - Working Set Size - WorkingSetSize: 226693120, PeakWorkingSetSize: 226693120, PageFaultCount: 59475 * Dump of thread ID 224 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 88906248.000000, User Time: 62790467584.000000, Wait Time: 1009484.000000 - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 - Registers - eax=00000007 ebx=0f270908 ecx=00000000 edx=10473da4 esi=0f52baa0 edi=0f253c20 eip=006c6d27 esp=0f104900 ebp=0f2708f8 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0f2708f8 0064a92b 0f251ae0 00000488 0f2e2dcc 0064a92b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006c6d27' 0f2e2db8 00455431 0f52b618 00000122 0f525b78 0f4e2690 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0064a92b' 0f4b915c 0044f0de 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00455431' 0f6d5e78 0065501d 40a5032b 40afc9c2 100cddf8 0065501d !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0044f0de' 100cdde8 005df71b 00000002 00000002 10332864 005df71b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0065501d' 10332858 005da7e1 10332d44 3de20020 005da7e1 1132f8dc !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005df71b' 10332d84 005dc121 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005da7e1' 10333188 007061a7 1031e8d0 10317b50 1132f7d8 007061a7 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005dc121' 1132f7c8 006f3667 1132f8d8 1132f8fc 1132fab8 006f3667 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007061a7' 1132faa8 004083f3 04600000 01000000 00000001 004083f3 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006f3667' 1132fb10 00408130 1132fb24 001324ef 1132fb24 00132451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004083f3' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 80000004 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 3760 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 156250.000000, User Time: 312500.000000, Wait Time: 1009478.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=3404ff74 eip=7d61ccc6 esp=3404ff38 ebp=3404ff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3404ff9c 7d4d14ef 00000064 00000000 3404ffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 3404ffac 00749924 00000064 3404ffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 3404ffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 3404ffec 00000000 00749910 00000000 00000000 34560000 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=996, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:18:28 (2684): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1872, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Jan 2014 10:17:20	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	1,036,800	1,305,291	1.2590
18 Jan 2014 00:35:12	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	1,010,880	1,273,789	1.2601
17 Jan 2014 15:46:16	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	984,960	1,242,519	1.2615
17 Jan 2014 07:01:48	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	959,040	1,211,426	1.2632
16 Jan 2014 23:04:36	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	933,120	1,180,506	1.2651
16 Jan 2014 13:39:35	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	907,200	1,148,748	1.2663
16 Jan 2014 04:17:30	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	881,280	1,114,422	1.2645
15 Jan 2014 18:24:38	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	855,360	1,079,379	1.2619
15 Jan 2014 08:28:51	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	829,440	1,043,951	1.2586
14 Jan 2014 22:21:18	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	803,520	1,011,031	1.2583
14 Jan 2014 12:44:14	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	777,600	977,485	1.2571
14 Jan 2014 02:57:03	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	751,680	943,005	1.2545
14 Jan 2014 00:41:08	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	725,760	911,882	1.2565
13 Jan 2014 09:12:54	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	699,840	880,934	1.2588
13 Jan 2014 00:36:03	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	673,920	849,940	1.2612
12 Jan 2014 15:45:30	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	648,000	818,820	1.2636
12 Jan 2014 07:13:06	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	622,080	786,772	1.2647
11 Jan 2014 21:05:40	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	596,160	753,032	1.2631
11 Jan 2014 12:13:29	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	570,240	719,338	1.2615
11 Jan 2014 02:04:48	1259513	16194421	hadcm3n_od6v_1900_40_008472218_3	544,320	686,334	1.2609