climateprediction.net home page
Task 12467480

Task 12467480

Name famous_x95i_1199_200_007073693_2
Workunit 7276993
Created 7 Jan 2011, 14:51:34 UTC
Sent 7 Jan 2011, 15:15:13 UTC
Report deadline 8 Apr 2011, 22:42:24 UTC
Received 11 Jan 2011, 21:20:24 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1123076
Run time 10 hours 27 min 17 sec
CPU time 9 hours 16 min 13 sec
Validate state Invalid
Credit 525.07
Device peak FLOPS 2.63 GFLOPS
Application version UK Met Office FAMOUS v6.11
i686-apple-darwin
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Sun Jan  9 13:16:07 2011

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9955346b    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc44c edx: 0x994ed0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc488 esp: 0xbfffc44c
   ss: 0x0000001f efl: 0x00000206 eip: 0x994ed0fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libirc.dylib
0x90162000 - 0x90165fff /usr/lib/system/libmathCommon.A.dylib
0x98403000 - 0x98411fff /usr/lib/libz.1.dylib
0x994ec000 - 0x99693fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34022, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Jan 10 00:17:10 2011

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9955346b    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc44c edx: 0x994ed0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc488 esp: 0xbfffc44c
   ss: 0x0000001f efl: 0x00000206 eip: 0x994ed0fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libirc.dylib
0x90162000 - 0x90165fff /usr/lib/system/libmathCommon.A.dylib
0x98403000 - 0x98411fff /usr/lib/libz.1.dylib
0x994ec000 - 0x99693fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=41528, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Jan 10 17:05:02 2011

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9955346b    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc44c edx: 0x994ed0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc488 esp: 0xbfffc44c
   ss: 0x0000001f efl: 0x00000206 eip: 0x994ed0fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libirc.dylib
0x90162000 - 0x90165fff /usr/lib/system/libmathCommon.A.dylib
0x98403000 - 0x98411fff /usr/lib/libz.1.dylib
0x994ec000 - 0x99693fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=52365, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Jan 10 18:21:21 2011

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9955346b    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc44c edx: 0x994ed0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc488 esp: 0xbfffc44c
   ss: 0x0000001f efl: 0x00000206 eip: 0x994ed0fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libirc.dylib
0x90162000 - 0x90165fff /usr/lib/system/libmathCommon.A.dylib
0x98403000 - 0x98411fff /usr/lib/libz.1.dylib
0x994ec000 - 0x99693fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=53182, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Jan 11 11:56:42 2011

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9955346b    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc44c edx: 0x994ed0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc488 esp: 0xbfffc44c
   ss: 0x0000001f efl: 0x00000206 eip: 0x994ed0fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libirc.dylib
0x90162000 - 0x90165fff /usr/lib/system/libmathCommon.A.dylib
0x98403000 - 0x98411fff /usr/lib/libz.1.dylib
0x994ec000 - 0x99693fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=65080, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Jan 11 11:56:44 2011

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x9955346b    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc44c edx: 0x994ed0fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc488 esp: 0xbfffc44c
   ss: 0x0000001f efl: 0x00000206 eip: 0x994ed0fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_x95i_1199_200_007073693/lib/libirc.dylib
0x90162000 - 0x90165fff /usr/lib/system/libmathCommon.A.dylib
0x98403000 - 0x98411fff /usr/lib/libz.1.dylib
0x994ec000 - 0x99693fff /usr/lib/libSystem.B.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=65080, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  
Sorry, too many model crashes! :-(
 (65748): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
11 Jan 2011 20:21:10 1123076 12467480 famous_x95i_1199_200_007073693_2 159,146 31,661 0.1989
11 Jan 2011 19:06:13 1123076 12467480 famous_x95i_1199_200_007073693_2 149,786 29,764 0.1987
11 Jan 2011 18:52:55 1123076 12467480 famous_x95i_1199_200_007073693_2 140,426 27,889 0.1986
11 Jan 2011 16:29:00 1123076 12467480 famous_x95i_1199_200_007073693_2 131,066 26,030 0.1986
11 Jan 2011 15:43:39 1123076 12467480 famous_x95i_1199_200_007073693_2 121,706 24,170 0.1986
11 Jan 2011 15:01:33 1123076 12467480 famous_x95i_1199_200_007073693_2 112,346 22,310 0.1986
11 Jan 2011 14:29:55 1123076 12467480 famous_x95i_1199_200_007073693_2 102,986 20,455 0.1986
11 Jan 2011 13:52:37 1123076 12467480 famous_x95i_1199_200_007073693_2 93,626 18,595 0.1986
11 Jan 2011 13:25:16 1123076 12467480 famous_x95i_1199_200_007073693_2 84,266 16,738 0.1986
11 Jan 2011 12:41:56 1123076 12467480 famous_x95i_1199_200_007073693_2 74,906 14,860 0.1984
11 Jan 2011 11:57:42 1123076 12467480 famous_x95i_1199_200_007073693_2 65,546 12,976 0.1980
11 Jan 2011 11:25:21 1123076 12467480 famous_x95i_1199_200_007073693_2 56,186 11,142 0.1983
11 Jan 2011 11:17:31 1123076 12467480 famous_x95i_1199_200_007073693_2 46,826 9,302 0.1987
11 Jan 2011 11:17:31 1123076 12467480 famous_x95i_1199_200_007073693_2 37,466 7,457 0.1990
11 Jan 2011 11:17:31 1123076 12467480 famous_x95i_1199_200_007073693_2 28,106 5,621 0.2000
11 Jan 2011 11:17:31 1123076 12467480 famous_x95i_1199_200_007073693_2 18,746 3,788 0.2021
11 Jan 2011 11:17:31 1123076 12467480 famous_x95i_1199_200_007073693_2 9,386 1,901 0.2025


©2024 climateprediction.net