Task 16157004

Name	hadcm3n_n4g5_1880_40_008375543_4
Workunit	8526402
Created	23 Dec 2013, 18:44:07 UTC
Sent	23 Dec 2013, 18:44:11 UTC
Report deadline	25 Mar 2014, 2:11:22 UTC
Received	8 Jan 2014, 21:16:06 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1259513
Run time	14 days 12 hours 3 min 34 sec
CPU time	13 days 23 hours 29 min 35 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> Unhandled Exception Detected... - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x00413067 Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 12/23/13 16:59:05 Install Directory : Data Directory : D:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_n4g5_1880_40_008375543;D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 280438, Write: 0, Other 606733 - I/O Transfers Counters - Read: 0, Write: 19241, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124432, QuotaPeakPagedPoolUsage: 124432 QuotaNonPagedPoolUsage: 5712, QuotaPeakNonPagedPoolUsage: 5792 - Virtual Memory Usage - VirtualSize: 1051377664, PeakVirtualSize: 1051377664 - Pagefile Usage - PagefileUsage: 231960576, PeakPagefileUsage: 231960576 - Working Set Size - WorkingSetSize: 227000320, PeakWorkingSetSize: 227000320, PageFaultCount: 62735 * Dump of thread ID 1224 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 258906256.000000, User Time: 187836088320.000000, Wait Time: 6080073.000000 - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x00413067 - Registers - eax=0ff829b4 ebx=0fe8031c ecx=0ff025b4 edx=39d0d7a4 esi=00000000 edi=39c8d3a4 eip=00413067 esp=0fc68da0 ebp=0fe80308 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 0fe80308 00429110 3ee3c946 3ef4d59c 3f03266e 1032c1b8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00413067' 1032c1a8 004408d5 0080f4e8 00000002 103332fc 004408d5 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00429110' 103332e8 0070640f 3df87738 3df88738 3df889c0 1132f7d8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004408d5' 1132f7c8 006f3667 1132f8d8 1132f8fc 1132fab8 006f3667 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0070640f' 1132faa8 004083f3 04600000 01000000 00000001 004083f3 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006f3667' 1132fb10 00408130 1132fb24 001324ef 1132fb24 00132451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004083f3' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 80000004 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 2276 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6080067.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=3404ff74 eip=7d61ccc6 esp=3404ff38 ebp=3404ff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3404ff9c 7d4d14ef 00000064 00000000 3404ffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 3404ffac 00749924 00000064 3404ffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 3404ffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 3404ffec 00000000 00749910 00000000 00000000 34560000 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1556, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1556, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:05:23 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:06:09 (2100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jan 2014 21:18:48	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	1,036,800	1,207,322	1.1645
08 Jan 2014 11:11:32	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	1,010,880	1,173,931	1.1613
08 Jan 2014 01:38:41	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	984,960	1,140,620	1.1580
07 Jan 2014 16:03:46	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	959,040	1,105,931	1.1532
07 Jan 2014 05:52:16	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	933,120	1,071,098	1.1479
06 Jan 2014 14:12:16	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	907,200	1,035,908	1.1419
06 Jan 2014 04:27:28	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	881,280	1,001,410	1.1363
05 Jan 2014 14:46:38	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	855,360	968,771	1.1326
05 Jan 2014 05:09:14	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	829,440	935,045	1.1273
04 Jan 2014 19:51:45	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	803,520	903,497	1.1244
04 Jan 2014 07:09:36	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	777,600	871,360	1.1206
03 Jan 2014 21:41:41	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	751,680	839,887	1.1173
03 Jan 2014 13:31:57	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	725,760	808,479	1.1140
03 Jan 2014 04:14:37	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	699,840	775,470	1.1081
02 Jan 2014 15:34:39	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	673,920	742,904	1.1024
02 Jan 2014 05:34:44	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	648,000	710,315	1.0962
01 Jan 2014 20:16:59	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	622,080	676,961	1.0882
01 Jan 2014 11:04:55	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	596,160	645,552	1.0829
31 Dec 2013 22:26:48	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	570,240	611,194	1.0718
31 Dec 2013 13:05:50	1259513	16157004	hadcm3n_n4g5_1880_40_008375543_4	544,320	578,754	1.0633