climateprediction.net (CPDN) home page
Task 11731544

Task 11731544

Name famous_v6ml_1399_200_006694355_0
Workunit 6897608
Created 26 Aug 2010, 16:06:37 UTC
Sent 13 Dec 2010, 15:16:23 UTC
Report deadline 14 Mar 2011, 22:43:34 UTC
Received 15 Dec 2010, 9:08:22 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1074937
Run time 16 hours 37 min 31 sec
CPU time 15 hours 24 min 4 sec
Validate state Invalid
Credit 1,111.83
Device peak FLOPS 3.12 GFLOPS
Application version UK Met Office FAMOUS v6.11
i686-apple-darwin
Stderr 
<core_client_version>6.10.56</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 08:36:01 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9900, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 11:57:31 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11643, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 11:57:33 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11643, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 11:57:35 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11643, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 12:35:48 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11984, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 12:46:21 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12123, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 13:20:19 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12498, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 13:30:44 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12612, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 15:04:20 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16704, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 17:26:07 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 17:49:15 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18543, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 18:09:24 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18764, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 18:14:55 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18826, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Mon Dec 13 22:45:40 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21190, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Dec 14 04:57:16 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24361, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Dec 14 04:57:18 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24361, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Tue Dec 14 08:29:49 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd4ac edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4e8 esp: 0xbfffd4ac
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26275, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.5 build 10H574
Wed Dec 15 00:41:19 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x921d646b    3  0xffffffff    4  0x013596b3    5  0x0135844f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc41c edx: 0x921700fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc458 esp: 0xbfffc41c
   ss: 0x0000001f efl: 0x00000206 eip: 0x921700fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12b4000 -  0x12cffff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libsvml.dylib
 0x1327000 -  0x13befff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libifcoremt.dylib
 0x1407000 -  0x1561fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libimf.dylib
 0x1674000 -  0x16a2fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v6ml_1399_200_006694355/lib/libirc.dylib
0x90160000 - 0x90163fff /usr/lib/system/libmathCommon.A.dylib
0x9216f000 - 0x92316fff /usr/lib/libSystem.B.dylib
0x95534000 - 0x95542fff /usr/lib/libz.1.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1128, iMonCtr=1
Model crash detected, will try to restart...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  
Sorry, too many model crashes! :-(
 (1128): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Dec 2010 10:02:37 1074937 11731544 famous_v6ml_1399_200_006694355_0 336,986 54,211 0.1609
14 Dec 2010 15:14:40 1074937 11731544 famous_v6ml_1399_200_006694355_0 327,626 52,679 0.1608
14 Dec 2010 14:01:28 1074937 11731544 famous_v6ml_1399_200_006694355_0 318,266 51,154 0.1607
14 Dec 2010 13:34:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 308,906 49,631 0.1607
14 Dec 2010 13:12:46 1074937 11731544 famous_v6ml_1399_200_006694355_0 299,546 48,109 0.1606
14 Dec 2010 12:40:23 1074937 11731544 famous_v6ml_1399_200_006694355_0 290,186 46,586 0.1605
14 Dec 2010 12:27:59 1074937 11731544 famous_v6ml_1399_200_006694355_0 280,826 45,063 0.1605
14 Dec 2010 12:02:18 1074937 11731544 famous_v6ml_1399_200_006694355_0 271,466 43,539 0.1604
14 Dec 2010 11:22:03 1074937 11731544 famous_v6ml_1399_200_006694355_0 262,106 42,014 0.1603
14 Dec 2010 10:55:45 1074937 11731544 famous_v6ml_1399_200_006694355_0 252,746 40,491 0.1602
14 Dec 2010 10:32:09 1074937 11731544 famous_v6ml_1399_200_006694355_0 243,386 38,971 0.1601
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 234,026 37,449 0.1600
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 224,666 35,929 0.1599
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 215,306 34,405 0.1598
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 205,946 32,882 0.1597
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 196,586 31,359 0.1595
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 187,226 29,834 0.1593
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 177,866 28,310 0.1592
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 168,506 26,786 0.1590
14 Dec 2010 10:24:13 1074937 11731544 famous_v6ml_1399_200_006694355_0 159,146 25,262 0.1587


©2025 cpdn.org