climateprediction.net home page
Task 16064395

Task 16064395

Name hadcm3n_n7pj_1960_40_008378357_2
Workunit 8529216
Created 10 Oct 2013, 13:36:34 UTC
Sent 10 Oct 2013, 13:36:45 UTC
Report deadline 9 Jan 2014, 21:03:56 UTC
Received 28 Oct 2013, 10:50:44 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1275748
Run time 12 days 15 hours 0 min 18 sec
CPU time 12 days 12 hours 22 min 5 sec
Validate state Valid
Credit 12,441.60
Device peak FLOPS 3.45 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>7.0.64</core_client_version>
<![CDATA[
<stderr_txt>
 Name          : avast! Antivirus
    Product Version       : 8.0.1497.376

ModLoad: 757e0000 00110000 C:\Windows\syswow64\KERNEL32.dll (6.1.7601.18229) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.18015

ModLoad: 75470000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.18229) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.18015

ModLoad: 754c0000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 75750000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 770b0000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.18177) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7601.18177 (win7sp1_gdr.130605-1534)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18177

ModLoad: 75cb0000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.18009) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.18009 (win7sp1_gdr.121121-1431)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.18009

ModLoad: 75b40000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 75390000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.18247) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 75d50000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 76e20000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.18205) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 751a0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17940) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.17940 (win7sp1_gdr.120824-0334)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17940

ModLoad: 75190000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6cda0000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74050000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 6ba40000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 74040000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7601.18177) (-exported- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 6.1.7601.18177 (win7sp1_gdr.130605-1534)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18177

ModLoad: 759a0000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 75c80000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

Get Product Name Failed.
ModLoad: 755c0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded)
    Linked PDB Filename   : oleaut32.pdb
    File Version          : 6.1.7601.17676
    Company Name          : Microsoft Corporation
    Product Name          : 
    Product Version       : 6.1.7601.17676

ModLoad: 75230000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 75730000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

ModLoad: 72bf0000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : dwmapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 76010000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 6.1.7601.17787 (win7sp1_gdr.120229-1502)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17787

ModLoad: 756c0000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 77280000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 72ee0000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 750c0000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 5464 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7547C41F

- Registers -
eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=7547c41f esp=1132f9c8 ebp=1132fa18
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!RaiseException+0x0 
1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad8 00408130 1132faec 114f4041 1132faec 114f3fc9 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff88 757f336a 7efde000 1132ffd4 77789f72 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 77789f72 7efde000 6030921b 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
1132ffd4 77789f45 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4676, iMonCtr=1
Model crash detected, will try to restart...
17:21:02 (4676): No heartbeat from core client for 30 sec - exiting
17:21:03 (4676): No heartbeat from core client for 30 sec - exiting
17:21:04 (4676): No heartbeat from core client for 30 sec - exiting
17:21:05 (4676): No heartbeat from core client for 30 sec - exiting
17:21:06 (4676): No heartbeat from core client for 30 sec - exiting
17:21:07 (4676): No heartbeat from core client for 30 sec - exiting
17:21:08 (4676): No heartbeat from core client for 30 sec - exiting
17:21:09 (4676): No heartbeat from core client for 30 sec - exiting
17:21:10 (4676): No heartbeat from core client for 30 sec - exiting
17:21:11 (4676): No heartbeat from core client for 30 sec - exiting
17:21:12 (4676): No heartbeat from core client for 30 sec - exiting
17:21:13 (4676): No heartbeat from core client for 30 sec - exiting
17:21:14 (4676): No heartbeat from core client for 30 sec - exiting
17:21:15 (4676): No heartbeat from core client for 30 sec - exiting
17:21:16 (4676): No heartbeat from core client for 30 sec - exiting
17:21:17 (4676): No heartbeat from core client for 30 sec - exiting
17:21:18 (4676): No heartbeat from core client for 30 sec - exiting
17:21:19 (4676): No heartbeat from core client for 30 sec - exiting
17:21:20 (4676): No heartbeat from core client for 30 sec - exiting
17:21:21 (4676): No heartbeat from core client for 30 sec - exiting
17:21:22 (4676): No heartbeat from core client for 30 sec - exiting
17:21:23 (4676): No heartbeat from core client for 30 sec - exiting
17:21:24 (4676): No heartbeat from core client for 30 sec - exiting
17:21:25 (4676): No heartbeat from core client for 30 sec - exiting
17:21:26 (4676): No heartbeat from core client for 30 sec - exiting
17:21:27 (4676): No heartbeat from core client for 30 sec - exiting
17:21:28 (4676): No heartbeat from core client for 30 sec - exiting
17:21:29 (4676): No heartbeat from core client for 30 sec - exiting
17:21:30 (4676): No heartbeat from core client for 30 sec - exiting
17:21:31 (4676): No heartbeat from core client for 30 sec - exiting
17:21:32 (4676): No heartbeat from core client for 30 sec - exiting
17:21:33 (4676): No heartbeat from core client for 30 sec - exiting
17:21:34 (4676): No heartbeat from core client for 30 sec - exiting
17:21:35 (4676): No heartbeat from core client for 30 sec - exiting
17:21:36 (4676): No heartbeat from core client for 30 sec - exiting
17:21:37 (4676): No heartbeat from core client for 30 sec - exiting
17:21:38 (4676): No heartbeat from core client for 30 sec - exiting
17:21:39 (4676): No heartbeat from core client for 30 sec - exiting
17:21:40 (4676): No heartbeat from core client for 30 sec - exiting
17:21:41 (4676): No heartbeat from core client for 30 sec - exiting
17:21:42 (4676): No heartbeat from core client for 30 sec - exiting
17:21:43 (4676): No heartbeat from core client for 30 sec - exiting
17:21:44 (4676): No heartbeat from core client for 30 sec - exiting
17:21:45 (4676): No heartbeat from core client for 30 sec - exiting
17:21:46 (4676): No heartbeat from core client for 30 sec - exiting
17:21:47 (4676): No heartbeat from core client for 30 sec - exiting
17:21:48 (4676): No heartbeat from core client for 30 sec - exiting
17:21:49 (4676): No heartbeat from core client for 30 sec - exiting
17:21:50 (4676): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:22:56 (4008): No heartbeat from core client for 30 sec - exiting
17:22:58 (4008): No heartbeat from core client for 30 sec - exiting
17:22:59 (4008): No heartbeat from core client for 30 sec - exiting
17:23:00 (4008): No heartbeat from core client for 30 sec - exiting
17:23:01 (4008): No heartbeat from core client for 30 sec - exiting
17:23:02 (4008): No heartbeat from core client for 30 sec - exiting
17:23:03 (4008): No heartbeat from core client for 30 sec - exiting
17:23:04 (4008): No heartbeat from core client for 30 sec - exiting
17:23:05 (4008): No heartbeat from core client for 30 sec - exiting
17:23:06 (4008): No heartbeat from core client for 30 sec - exiting
17:23:07 (4008): No heartbeat from core client for 30 sec - exiting
17:23:08 (4008): No heartbeat from core client for 30 sec - exiting
17:23:09 (4008): No heartbeat from core client for 30 sec - exiting
17:23:10 (4008): No heartbeat from core client for 30 sec - exiting
17:23:11 (4008): No heartbeat from core client for 30 sec - exiting
17:23:12 (4008): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:23:59 (5548): No heartbeat from core client for 30 sec - exiting
17:24:00 (5548): No heartbeat from core client for 30 sec - exiting
17:24:01 (5548): No heartbeat from core client for 30 sec - exiting
17:24:02 (5548): No heartbeat from core client for 30 sec - exiting
17:24:03 (5548): No heartbeat from core client for 30 sec - exiting
17:24:04 (5548): No heartbeat from core client for 30 sec - exiting
17:24:05 (5548): No heartbeat from core client for 30 sec - exiting
17:24:06 (5548): No heartbeat from core client for 30 sec - exiting
17:24:07 (5548): No heartbeat from core client for 30 sec - exiting
17:24:08 (5548): No heartbeat from core client for 30 sec - exiting
17:24:09 (5548): No heartbeat from core client for 30 sec - exiting
17:24:10 (5548): No heartbeat from core client for 30 sec - exiting
17:24:11 (5548): No heartbeat from core client for 30 sec - exiting
17:24:12 (5548): No heartbeat from core client for 30 sec - exiting
17:24:13 (5548): No heartbeat from core client for 30 sec - exiting
17:24:14 (5548): No heartbeat from core client for 30 sec - exiting
17:24:15 (5548): No heartbeat from core client for 30 sec - exiting
17:24:16 (5548): No heartbeat from core client for 30 sec - exiting
17:24:17 (5548): No heartbeat from core client for 30 sec - exiting
17:24:18 (5548): No heartbeat from core client for 30 sec - exiting
17:24:19 (5548): No heartbeat from core client for 30 sec - exiting
17:24:20 (5548): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:24:21 (5548): No heartbeat from core client for 30 sec - exiting
17:25:09 (4440): No heartbeat from core client for 30 sec - exiting
17:25:10 (4440): No heartbeat from core client for 30 sec - exiting
17:25:11 (4440): No heartbeat from core client for 30 sec - exiting
17:25:12 (4440): No heartbeat from core client for 30 sec - exiting
17:25:13 (4440): No heartbeat from core client for 30 sec - exiting
17:25:14 (4440): No heartbeat from core client for 30 sec - exiting
17:25:15 (4440): No heartbeat from core client for 30 sec - exiting
17:25:16 (4440): No heartbeat from core client for 30 sec - exiting
17:25:17 (4440): No heartbeat from core client for 30 sec - exiting
17:25:18 (4440): No heartbeat from core client for 30 sec - exiting
17:25:19 (4440): No heartbeat from core client for 30 sec - exiting
17:25:20 (4440): No heartbeat from core client for 30 sec - exiting
17:25:21 (4440): No heartbeat from core client for 30 sec - exiting
17:25:22 (4440): No heartbeat from core client for 30 sec - exiting
17:25:23 (4440): No heartbeat from core client for 30 sec - exiting
17:25:24 (4440): No heartbeat from core client for 30 sec - exiting
17:25:25 (4440): No heartbeat from core client for 30 sec - exiting
17:25:26 (4440): No heartbeat from core client for 30 sec - exiting
17:25:27 (4440): No heartbeat from core client for 30 sec - exiting
17:25:28 (4440): No heartbeat from core client for 30 sec - exiting
17:25:29 (4440): No heartbeat from core client for 30 sec - exiting
17:25:30 (4440): No heartbeat from core client for 30 sec - exiting
17:25:31 (4440): No heartbeat from core client for 30 sec - exiting
17:25:32 (4440): No heartbeat from core client for 30 sec - exiting
17:25:33 (4440): No heartbeat from core client for 30 sec - exiting
17:25:34 (4440): No heartbeat from core client for 30 sec - exiting
17:25:35 (4440): No heartbeat from core client for 30 sec - exiting
17:25:36 (4440): No heartbeat from core client for 30 sec - exiting
17:25:37 (4440): No heartbeat from core client for 30 sec - exiting
17:25:38 (4440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:25:39 (4440): No heartbeat from core client for 30 sec - exiting
17:26:36 (4408): No heartbeat from core client for 30 sec - exiting
17:26:37 (4408): No heartbeat from core client for 30 sec - exiting
17:26:38 (4408): No heartbeat from core client for 30 sec - exiting
17:26:39 (4408): No heartbeat from core client for 30 sec - exiting
17:26:40 (4408): No heartbeat from core client for 30 sec - exiting
17:26:41 (4408): No heartbeat from core client for 30 sec - exiting
17:26:42 (4408): No heartbeat from core client for 30 sec - exiting
17:26:43 (4408): No heartbeat from core client for 30 sec - exiting
17:26:44 (4408): No heartbeat from core client for 30 sec - exiting
17:26:45 (4408): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:27:27 (5632): No heartbeat from core client for 30 sec - exiting
17:27:28 (5632): No heartbeat from core client for 30 sec - exiting
17:27:29 (5632): No heartbeat from core client for 30 sec - exiting
17:27:30 (5632): No heartbeat from core client for 30 sec - exiting
17:27:31 (5632): No heartbeat from core client for 30 sec - exiting
17:27:32 (5632): No heartbeat from core client for 30 sec - exiting
17:27:33 (5632): No heartbeat from core client for 30 sec - exiting
17:27:34 (5632): No heartbeat from core client for 30 sec - exiting
17:27:35 (5632): No heartbeat from core client for 30 sec - exiting
17:27:36 (5632): No heartbeat from core client for 30 sec - exiting
17:27:37 (5632): No heartbeat from core client for 30 sec - exiting
17:27:38 (5632): No heartbeat from core client for 30 sec - exiting
17:27:39 (5632): No heartbeat from core client for 30 sec - exiting
17:27:40 (5632): No heartbeat from core client for 30 sec - exiting
17:27:41 (5632): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:28:19 (7016): No heartbeat from core client for 30 sec - exiting
17:28:20 (7016): No heartbeat from core client for 30 sec - exiting
17:28:21 (7016): No heartbeat from core client for 30 sec - exiting
17:28:22 (7016): No heartbeat from core client for 30 sec - exiting
17:28:23 (7016): No heartbeat from core client for 30 sec - exiting
17:28:24 (7016): No heartbeat from core client for 30 sec - exiting
17:28:25 (7016): No heartbeat from core client for 30 sec - exiting
17:28:26 (7016): No heartbeat from core client for 30 sec - exiting
17:28:27 (7016): No heartbeat from core client for 30 sec - exiting
17:28:28 (7016): No heartbeat from core client for 30 sec - exiting
17:28:29 (7016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:29:07 (6796): No heartbeat from core client for 30 sec - exiting
17:29:09 (6796): No heartbeat from core client for 30 sec - exiting
17:29:10 (6796): No heartbeat from core client for 30 sec - exiting
17:29:11 (6796): No heartbeat from core client for 30 sec - exiting
17:29:12 (6796): No heartbeat from core client for 30 sec - exiting
17:29:13 (6796): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:29:14 (6796): No heartbeat from core client for 30 sec - exiting
17:29:53 (3672): No heartbeat from core client for 30 sec - exiting
17:29:54 (3672): No heartbeat from core client for 30 sec - exiting
17:29:55 (3672): No heartbeat from core client for 30 sec - exiting
17:29:56 (3672): No heartbeat from core client for 30 sec - exiting
17:29:57 (3672): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:30:35 (6596): No heartbeat from core client for 30 sec - exiting
17:30:36 (6596): No heartbeat from core client for 30 sec - exiting
17:30:37 (6596): No heartbeat from core client for 30 sec - exiting
17:30:38 (6596): No heartbeat from core client for 30 sec - exiting
17:30:39 (6596): No heartbeat from core client for 30 sec - exiting
17:30:40 (6596): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:31:41 (6176): No heartbeat from core client for 30 sec - exiting
17:31:42 (6176): No heartbeat from core client for 30 sec - exiting
17:31:43 (6176): No heartbeat from core client for 30 sec - exiting
17:31:44 (6176): No heartbeat from core client for 30 sec - exiting
17:31:45 (6176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4320, iMonCtr=1
Model crash detected, will try to restart...
17:32:47 (4320): No heartbeat from core client for 30 sec - exiting
17:32:48 (4320): No heartbeat from core client for 30 sec - exiting
17:32:49 (4320): No heartbeat from core client for 30 sec - exiting
17:32:50 (4320): No heartbeat from core client for 30 sec - exiting
17:32:51 (4320): No heartbeat from core client for 30 sec - exiting
17:32:52 (4320): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:32:53 (4320): No heartbeat from core client for 30 sec - exiting
17:33:32 (6296): No heartbeat from core client for 30 sec - exiting
17:33:33 (6296): No heartbeat from core client for 30 sec - exiting
17:33:34 (6296): No heartbeat from core client for 30 sec - exiting
17:33:35 (6296): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:34:12 (4772): No heartbeat from core client for 30 sec - exiting
17:34:13 (4772): No heartbeat from core client for 30 sec - exiting
17:34:14 (4772): No heartbeat from core client for 30 sec - exiting
17:34:15 (4772): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:34:53 (3344): No heartbeat from core client for 30 sec - exiting
17:34:54 (3344): No heartbeat from core client for 30 sec - exiting
17:34:55 (3344): No heartbeat from core client for 30 sec - exiting
17:34:56 (3344): No heartbeat from core client for 30 sec - exiting
17:34:57 (3344): No heartbeat from core client for 30 sec - exiting
17:34:58 (3344): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:35:35 (4200): No heartbeat from core client for 30 sec - exiting
17:35:36 (4200): No heartbeat from core client for 30 sec - exiting
17:35:37 (4200): No heartbeat from core client for 30 sec - exiting
17:35:38 (4200): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:36:20 (6864): No heartbeat from core client for 30 sec - exiting
17:36:21 (6864): No heartbeat from core client for 30 sec - exiting
17:36:22 (6864): No heartbeat from core client for 30 sec - exiting
17:36:23 (6864): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7547C41F

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 10/23/13 17:37:24
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_n7pj_1960_40_008378357;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 77750000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.18247) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7601.18247 (win7sp1_gdr.130828-1532)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18247

ModLoad: 732a0000 0003f000 C:\Program Files\AVAST Software\Avast\snxhk.dll (8.0.1497.376) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\Avast800\src\BUILDS\Release\x86\snxhk.pdb
    File Version          : 8.0.1497.376
    Company Name          : AVAST Software
    Product Name          : avast! Antivirus
    Product Version       : 8.0.1497.376

Get Company Name Failed.
Get Product Name Failed.
ModLoad: 757e0000 00110000 C:\Windows\syswow64\KERNEL32.dll (6.1.7601.18229) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 
    Company Name          : 
    Product Name          : 
    Product Version       : 

ModLoad: 75470000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.18229) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.18229 (win7sp1_gdr.130801-1533)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18229

ModLoad: 754c0000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 75750000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 770b0000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.18177) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7601.18177 (win7sp1_gdr.130605-1534)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18177

ModLoad: 75cb0000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.18009) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.18009 (win7sp1_gdr.121121-1431)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.18009

ModLoad: 75b40000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 75390000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.18247) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7601.18247 (win7sp1_gdr.130828-1532)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18247

ModLoad: 75d50000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 76e20000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.18205) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 751a0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17940) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.17940 (win7sp1_gdr.120824-0334)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17940

ModLoad: 75190000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6cda0000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74050000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 6ba40000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 74040000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7601.18177) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 6.1.7601.18177 (win7sp1_gdr.130605-1534)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18177

ModLoad: 759a0000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 75c80000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

Get Product Name Failed.
ModLoad: 755c0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : oleaut32.pdb
    File Version          : 6.1.7601.17676
    Company Name          : Microsoft Corporation
    Product Name          : 
    Product Version       : 6.1.7601.17676

ModLoad: 75230000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 75730000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

ModLoad: 72bf0000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : dwmapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 76010000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 6.1.7601.17787 (win7sp1_gdr.120229-1502)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17787

ModLoad: 756c0000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 77280000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 72ee0000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 2064 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7547C41F

- Registers -
eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=7547c41f esp=1132f9c4 ebp=1132fa14
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 KERNELBASE!RaiseException+0x0 
1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad8 00408130 1132faec 11524041 1132faec 11523fc9 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff88 757f336a 7efde000 1132ffd4 77789f72 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 77789f72 7efde000 6020fc06 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
1132ffd4 77789f45 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 6608 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=33bfff48 edi=00000000
eip=7776fd91 esp=33bfff04 ebp=33bfff6c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
33bfff6c 75484498 00000064 00000000 33bfff88 00749924 ntdll!ZwDelayExecution+0x0 
33bfff7c 00749924 00000064 33bfff94 757f336a 00000000 KERNELBASE!Sleep+0x0 
33bfff88 757f336a 00000000 33bfffd4 77789f72 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 
33bfff94 77789f72 00000000 42adfc06 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
33bfffd4 77789f45 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
33bfffec 00000000 00749910 00000000 00000000 34110000 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1
Model crash detected, will try to restart...
17:38:11 (3164): No heartbeat from core client for 30 sec - exiting
17:38:12 (3164): No heartbeat from core client for 30 sec - exiting
17:38:13 (3164): No heartbeat from core client for 30 sec - exiting
17:38:14 (3164): No heartbeat from core client for 30 sec - exiting
17:38:15 (3164): No heartbeat from core client for 30 sec - exiting
17:38:16 (3164): No heartbeat from core client for 30 sec - exiting
17:38:17 (3164): No heartbeat from core client for 30 sec - exiting
17:38:18 (3164): No heartbeat from core client for 30 sec - exiting
17:38:19 (3164): No heartbeat from core client for 30 sec - exiting
17:38:20 (3164): No heartbeat from core client for 30 sec - exiting
17:38:21 (3164): No heartbeat from core client for 30 sec - exiting
17:38:22 (3164): No heartbeat from core client for 30 sec - exiting
17:38:23 (3164): No heartbeat from core client for 30 sec - exiting
17:38:24 (3164): No heartbeat from core client for 30 sec - exiting
17:38:25 (3164): No heartbeat from core client for 30 sec - exiting
17:38:26 (3164): No heartbeat from core client for 30 sec - exiting
17:38:27 (3164): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:39:44 (1572): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:40:18 (6332): No heartbeat from core client for 30 sec - exiting
17:40:20 (6332): No heartbeat from core client for 30 sec - exiting
17:40:21 (6332): No heartbeat from core client for 30 sec - exiting
17:40:22 (6332): No heartbeat from core client for 30 sec - exiting
17:40:23 (6332): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:40:24 (6332): No heartbeat from core client for 30 sec - exiting
17:41:01 (1180): No heartbeat from core client for 30 sec - exiting
17:41:02 (1180): No heartbeat from core client for 30 sec - exiting
17:41:03 (1180): No heartbeat from core client for 30 sec - exiting
17:41:04 (1180): No heartbeat from core client for 30 sec - exiting
17:41:05 (1180): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:41:44 (6324): No heartbeat from core client for 30 sec - exiting
17:41:45 (6324): No heartbeat from core client for 30 sec - exiting
17:41:46 (6324): No heartbeat from core client for 30 sec - exiting
17:41:47 (6324): No heartbeat from core client for 30 sec - exiting
17:41:48 (6324): No heartbeat from core client for 30 sec - exiting
17:41:49 (6324): No heartbeat from core client for 30 sec - exiting
17:41:50 (6324): No heartbeat from core client for 30 sec - exiting
17:41:51 (6324): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:42:39 (7152): No heartbeat from core client for 30 sec - exiting
17:42:40 (7152): No heartbeat from core client for 30 sec - exiting
17:42:41 (7152): No heartbeat from core client for 30 sec - exiting
17:42:42 (7152): No heartbeat from core client for 30 sec - exiting
17:42:43 (7152): No heartbeat from core client for 30 sec - exiting
17:42:44 (7152): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:43:21 (3048): No heartbeat from core client for 30 sec - exiting
17:43:22 (3048): No heartbeat from core client for 30 sec - exiting
17:43:23 (3048): No heartbeat from core client for 30 sec - exiting
17:43:24 (3048): No heartbeat from core client for 30 sec - exiting
17:43:25 (3048): No heartbeat from core client for 30 sec - exiting
17:43:26 (3048): No heartbeat from core client for 30 sec - exiting
17:43:27 (3048): No heartbeat from core client for 30 sec - exiting
17:43:28 (3048): No heartbeat from core client for 30 sec - exiting
17:43:29 (3048): No heartbeat from core client for 30 sec - exiting
17:43:30 (3048): No heartbeat from core client for 30 sec - exiting
17:43:31 (3048): No heartbeat from core client for 30 sec - exiting
17:43:32 (3048): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:44:09 (6968): No heartbeat from core client for 30 sec - exiting
17:44:10 (6968): No heartbeat from core client for 30 sec - exiting
17:44:11 (6968): No heartbeat from core client for 30 sec - exiting
17:44:12 (6968): No heartbeat from core client for 30 sec - exiting
17:44:13 (6968): No heartbeat from core client for 30 sec - exiting
17:44:14 (6968): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:44:15 (6968): No heartbeat from core client for 30 sec - exiting
17:44:54 (6944): No heartbeat from core client for 30 sec - exiting
17:44:55 (6944): No heartbeat from core client for 30 sec - exiting
17:44:56 (6944): No heartbeat from core client for 30 sec - exiting
17:44:57 (6944): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:45:35 (5644): No heartbeat from core client for 30 sec - exiting
17:45:36 (5644): No heartbeat from core client for 30 sec - exiting
17:45:37 (5644): No heartbeat from core client for 30 sec - exiting
17:45:38 (5644): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:46:17 (6724): No heartbeat from core client for 30 sec - exiting
17:46:18 (6724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:46:55 (252): No heartbeat from core client for 30 sec - exiting
17:46:56 (252): No heartbeat from core client for 30 sec - exiting
17:46:57 (252): No heartbeat from core client for 30 sec - exiting
17:46:58 (252): No heartbeat from core client for 30 sec - exiting
17:46:59 (252): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:47:57 (1116): No heartbeat from core client for 30 sec - exiting
17:47:58 (1116): No heartbeat from core client for 30 sec - exiting
17:47:59 (1116): No heartbeat from core client for 30 sec - exiting
17:48:00 (1116): No heartbeat from core client for 30 sec - exiting
17:48:01 (1116): No heartbeat from core client for 30 sec - exiting
17:48:02 (1116): No heartbeat from core client for 30 sec - exiting
17:48:03 (1116): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:48:04 (1116): No heartbeat from core client for 30 sec - exiting
17:48:40 (1920): No heartbeat from core client for 30 sec - exiting
17:48:41 (1920): No heartbeat from core client for 30 sec - exiting
17:48:42 (1920): No heartbeat from core client for 30 sec - exiting
17:48:43 (1920): No heartbeat from core client for 30 sec - exiting
17:48:44 (1920): No heartbeat from core client for 30 sec - exiting
17:48:45 (1920): No heartbeat from core client for 30 sec - exiting
17:48:46 (1920): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:49:26 (4928): No heartbeat from core client for 30 sec - exiting
17:49:27 (4928): No heartbeat from core client for 30 sec - exiting
17:49:28 (4928): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:50:01 (5300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:50:57 (5136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:51:37 (6084): No heartbeat from core client for 30 sec - exiting
17:51:39 (6084): No heartbeat from core client for 30 sec - exiting
17:51:40 (6084): No heartbeat from core client for 30 sec - exiting
17:51:41 (6084): No heartbeat from core client for 30 sec - exiting
17:51:42 (6084): No heartbeat from core client for 30 sec - exiting
17:51:43 (6084): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:51:44 (6084): No heartbeat from core client for 30 sec - exiting
17:52:24 (5000): No heartbeat from core client for 30 sec - exiting
17:52:25 (5000): No heartbeat from core client for 30 sec - exiting
17:52:26 (5000): No heartbeat from core client for 30 sec - exiting
17:52:27 (5000): No heartbeat from core client for 30 sec - exiting
17:52:28 (5000): No heartbeat from core client for 30 sec - exiting
17:52:29 (5000): No heartbeat from core client for 30 sec - exiting
17:52:30 (5000): No heartbeat from core client for 30 sec - exiting
17:52:31 (5000): No heartbeat from core client for 30 sec - exiting
17:52:32 (5000): No heartbeat from core client for 30 sec - exiting
17:52:33 (5000): No heartbeat from core client for 30 sec - exiting
17:52:34 (5000): No heartbeat from core client for 30 sec - exiting
17:52:35 (5000): No heartbeat from core client for 30 sec - exiting
17:52:36 (5000): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:53:50 (6256): No heartbeat from core client for 30 sec - exiting
17:53:51 (6256): No heartbeat from core client for 30 sec - exiting
17:53:52 (6256): No heartbeat from core client for 30 sec - exiting
17:53:53 (6256): No heartbeat from core client for 30 sec - exiting
17:53:54 (6256): No heartbeat from core client for 30 sec - exiting
17:53:55 (6256): No heartbeat from core client for 30 sec - exiting
17:53:56 (6256): No heartbeat from core client for 30 sec - exiting
17:53:57 (6256): No heartbeat from core client for 30 sec - exiting
17:53:58 (6256): No heartbeat from core client for 30 sec - exiting
17:53:59 (6256): No heartbeat from core client for 30 sec - exiting
17:54:00 (6256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:54:45 (5616): No heartbeat from core client for 30 sec - exiting
17:54:46 (5616): No heartbeat from core client for 30 sec - exiting
17:54:47 (5616): No heartbeat from core client for 30 sec - exiting
17:54:48 (5616): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:55:24 (6884): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:55:25 (6884): No heartbeat from core client for 30 sec - exiting
17:56:05 (4216): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:57:10 (1496): No heartbeat from core client for 30 sec - exiting
17:57:11 (1496): No heartbeat from core client for 30 sec - exiting
17:57:12 (1496): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:57:49 (5040): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:58:35 (4240): No heartbeat from core client for 30 sec - exiting
17:58:36 (4240): No heartbeat from core client for 30 sec - exiting
17:58:37 (4240): No heartbeat from core client for 30 sec - exiting
17:58:38 (4240): No heartbeat from core client for 30 sec - exiting
17:58:39 (4240): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:59:18 (7028): No heartbeat from core client for 30 sec - exiting
17:59:19 (7028): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:59:54 (6960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:00:44 (6840): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:01:25 (6424): No heartbeat from core client for 30 sec - exiting
18:01:26 (6424): No heartbeat from core client for 30 sec - exiting
18:01:27 (6424): No heartbeat from core client for 30 sec - exiting
18:01:28 (6424): No heartbeat from core client for 30 sec - exiting
18:01:29 (6424): No heartbeat from core client for 30 sec - exiting
18:01:30 (6424): No heartbeat from core client for 30 sec - exiting
18:01:31 (6424): No heartbeat from core client for 30 sec - exiting
18:01:32 (6424): No heartbeat from core client for 30 sec - exiting
18:01:33 (6424): No heartbeat from core client for 30 sec - exiting
18:01:34 (6424): No heartbeat from core client for 30 sec - exiting
18:01:35 (6424): No heartbeat from core client for 30 sec - exiting
18:01:36 (6424): No heartbeat from core client for 30 sec - exiting
18:01:37 (6424): No heartbeat from core client for 30 sec - exiting
18:01:38 (6424): No heartbeat from core client for 30 sec - exiting
18:01:39 (6424): No heartbeat from core client for 30 sec - exiting
18:01:40 (6424): No heartbeat from core client for 30 sec - exiting
18:01:41 (6424): No heartbeat from core client for 30 sec - exiting
18:01:42 (6424): No heartbeat from core client for 30 sec - exiting
18:01:43 (6424): No heartbeat from core client for 30 sec - exiting
18:01:44 (6424): No heartbeat from core client for 30 sec - exiting
18:01:45 (6424): No heartbeat from core client for 30 sec - exiting
18:01:46 (6424): No heartbeat from core client for 30 sec - exiting
18:01:47 (6424): No heartbeat from core client for 30 sec - exiting
18:01:48 (6424): No heartbeat from core client for 30 sec - exiting
18:01:49 (6424): No heartbeat from core client for 30 sec - exiting
18:01:50 (6424): No heartbeat from core client for 30 sec - exiting
18:01:51 (6424): No heartbeat from core client for 30 sec - exiting
18:01:52 (6424): No heartbeat from core client for 30 sec - exiting
18:01:53 (6424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:02:35 (5972): No heartbeat from core client for 30 sec - exiting
18:02:37 (5972): No heartbeat from core client for 30 sec - exiting
18:02:38 (5972): No heartbeat from core client for 30 sec - exiting
18:02:39 (5972): No heartbeat from core client for 30 sec - exiting
18:02:40 (5972): No heartbeat from core client for 30 sec - exiting
18:02:41 (5972): No heartbeat from core client for 30 sec - exiting
18:02:42 (5972): No heartbeat from core client for 30 sec - exiting
18:02:43 (5972): No heartbeat from core client for 30 sec - exiting
18:02:44 (5972): No heartbeat from core client for 30 sec - exiting
18:02:45 (5972): No heartbeat from core client for 30 sec - exiting
18:02:46 (5972): No heartbeat from core client for 30 sec - exiting
18:02:47 (5972): No heartbeat from core client for 30 sec - exiting
18:02:48 (5972): No heartbeat from core client for 30 sec - exiting
18:02:49 (5972): No heartbeat from core client for 30 sec - exiting
18:02:50 (5972): No heartbeat from core client for 30 sec - exiting
18:02:51 (5972): No heartbeat from core client for 30 sec - exiting
18:02:52 (5972): No heartbeat from core client for 30 sec - exiting
18:02:53 (5972): No heartbeat from core client for 30 sec - exiting
18:02:54 (5972): No heartbeat from core client for 30 sec - exiting
18:02:55 (5972): No heartbeat from core client for 30 sec - exiting
18:02:56 (5972): No heartbeat from core client for 30 sec - exiting
18:02:57 (5972): No heartbeat from core client for 30 sec - exiting
18:02:58 (5972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:03:59 (2664): No heartbeat from core client for 30 sec - exiting
18:04:00 (2664): No heartbeat from core client for 30 sec - exiting
18:04:01 (2664): No heartbeat from core client for 30 sec - exiting
18:04:02 (2664): No heartbeat from core client for 30 sec - exiting
18:04:03 (2664): No heartbeat from core client for 30 sec - exiting
18:04:04 (2664): No heartbeat from core client for 30 sec - exiting
18:04:05 (2664): No heartbeat from core client for 30 sec - exiting
18:04:06 (2664): No heartbeat from core client for 30 sec - exiting
18:04:07 (2664): No heartbeat from core client for 30 sec - exiting
18:04:08 (2664): No heartbeat from core client for 30 sec - exiting
18:04:09 (2664): No heartbeat from core client for 30 sec - exiting
18:04:10 (2664): No heartbeat from core client for 30 sec - exiting
18:04:11 (2664): No heartbeat from core client for 30 sec - exiting
18:04:12 (2664): No heartbeat from core client for 30 sec - exiting
18:04:13 (2664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:05:40 (4744): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:06:33 (5428): No heartbeat from core client for 30 sec - exiting
18:06:34 (5428): No heartbeat from core client for 30 sec - exiting
18:06:35 (5428): No heartbeat from core client for 30 sec - exiting
18:06:36 (5428): No heartbeat from core client for 30 sec - exiting
18:06:37 (5428): No heartbeat from core client for 30 sec - exiting
18:06:38 (5428): No heartbeat from core client for 30 sec - exiting
18:06:39 (5428): No heartbeat from core client for 30 sec - exiting
18:06:40 (5428): No heartbeat from core client for 30 sec - exiting
18:06:41 (5428): No heartbeat from core client for 30 sec - exiting
18:06:42 (5428): No heartbeat from core client for 30 sec - exiting
18:06:43 (5428): No heartbeat from core client for 30 sec - exiting
18:06:44 (5428): No heartbeat from core client for 30 sec - exiting
18:06:45 (5428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:07:32 (5340): No heartbeat from core client for 30 sec - exiting
18:07:33 (5340): No heartbeat from core client for 30 sec - exiting
18:07:34 (5340): No heartbeat from core client for 30 sec - exiting
18:07:35 (5340): No heartbeat from core client for 30 sec - exiting
18:07:36 (5340): No heartbeat from core client for 30 sec - exiting
18:07:37 (5340): No heartbeat from core client for 30 sec - exiting
18:07:38 (5340): No heartbeat from core client for 30 sec - exiting
18:07:39 (5340): No heartbeat from core client for 30 sec - exiting
18:07:40 (5340): No heartbeat from core client for 30 sec - exiting
18:07:41 (5340): No heartbeat from core client for 30 sec - exiting
18:07:42 (5340): No heartbeat from core client for 30 sec - exiting
18:07:43 (5340): No heartbeat from core client for 30 sec - exiting
18:07:44 (5340): No heartbeat from core client for 30 sec - exiting
18:07:45 (5340): No heartbeat from core client for 30 sec - exiting
18:07:46 (5340): No heartbeat from core client for 30 sec - exiting
18:07:47 (5340): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:08:27 (6828): No heartbeat from core client for 30 sec - exiting
18:08:28 (6828): No heartbeat from core client for 30 sec - exiting
18:08:29 (6828): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:09:22 (5136): No heartbeat from core client for 30 sec - exiting
18:09:23 (5136): No heartbeat from core client for 30 sec - exiting
18:09:24 (5136): No heartbeat from core client for 30 sec - exiting
18:09:25 (5136): No heartbeat from core client for 30 sec - exiting
18:09:26 (5136): No heartbeat from core client for 30 sec - exiting
18:09:27 (5136): No heartbeat from core client for 30 sec - exiting
18:09:28 (5136): No heartbeat from core client for 30 sec - exiting
18:09:29 (5136): No heartbeat from core client for 30 sec - exiting
18:09:30 (5136): No heartbeat from core client for 30 sec - exiting
18:09:31 (5136): No heartbeat from core client for 30 sec - exiting
18:09:32 (5136): No heartbeat from core client for 30 sec - exiting
18:09:33 (5136): No heartbeat from core client for 30 sec - exiting
18:09:34 (5136): No heartbeat from core client for 30 sec - exiting
18:09:35 (5136): No heartbeat from core client for 30 sec - exiting
18:09:36 (5136): No heartbeat from core client for 30 sec - exiting
18:09:37 (5136): No heartbeat from core client for 30 sec - exiting
18:09:38 (5136): No heartbeat from core client for 30 sec - exiting
18:09:39 (5136): No heartbeat from core client for 30 sec - exiting
18:09:40 (5136): No heartbeat from core client for 30 sec - exiting
18:09:41 (5136): No heartbeat from core client for 30 sec - exiting
18:09:42 (5136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:09:43 (5136): No heartbeat from core client for 30 sec - exiting
18:10:26 (6084): No heartbeat from core client for 30 sec - exiting
18:10:27 (6084): No heartbeat from core client for 30 sec - exiting
18:10:28 (6084): No heartbeat from core client for 30 sec - exiting
18:10:29 (6084): No heartbeat from core client for 30 sec - exiting
18:10:30 (6084): No heartbeat from core client for 30 sec - exiting
18:10:31 (6084): No heartbeat from core client for 30 sec - exiting
18:10:32 (6084): No heartbeat from core client for 30 sec - exiting
18:10:33 (6084): No heartbeat from core client for 30 sec - exiting
18:10:34 (6084): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:14:31 (3444): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:08:59 (860): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
28 Oct 2013 09:46:21 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 1,036,800 1,080,915 1.0425
28 Oct 2013 01:12:16 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 1,010,880 1,054,131 1.0428
27 Oct 2013 17:41:23 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 984,960 1,027,244 1.0429
27 Oct 2013 10:08:55 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 959,040 1,000,434 1.0432
27 Oct 2013 02:41:01 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 933,120 973,564 1.0433
26 Oct 2013 19:02:43 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 907,200 946,373 1.0432
26 Oct 2013 11:23:12 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 881,280 919,034 1.0428
26 Oct 2013 03:44:43 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 855,360 891,681 1.0425
25 Oct 2013 20:05:52 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 829,440 864,385 1.0421
25 Oct 2013 08:23:08 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 803,520 837,115 1.0418
25 Oct 2013 00:39:15 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 777,600 809,539 1.0411
24 Oct 2013 16:59:05 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 751,680 781,960 1.0403
24 Oct 2013 08:39:14 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 725,760 754,577 1.0397
24 Oct 2013 00:54:59 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 699,840 726,944 1.0387
23 Oct 2013 17:14:15 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 673,920 699,618 1.0381
23 Oct 2013 07:16:04 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 648,000 672,837 1.0383
22 Oct 2013 23:37:36 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 622,080 645,817 1.0382
22 Oct 2013 15:51:29 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 596,160 618,373 1.0373
22 Oct 2013 07:47:37 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 570,240 591,268 1.0369
21 Oct 2013 23:49:04 1275748 16064395 hadcm3n_n7pj_1960_40_008378357_2 544,320 564,451 1.0370


©2024 climateprediction.net