Name | famous_v2vd_1999_200_006689487_1 |
Workunit | 6892740 |
Created | 26 Aug 2010, 15:50:19 UTC |
Sent | 4 Sep 2010, 22:28:18 UTC |
Report deadline | 5 Dec 2010, 5:55:29 UTC |
Received | 12 Sep 2010, 9:18:46 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1072552 |
Run time | 2 days 14 hours 37 min 26 sec |
CPU time | 2 days 10 hours 48 min 9 sec |
Validate state | Invalid |
Credit | 5,002.91 |
Device peak FLOPS | 3.56 GFLOPS |
Application version | UK Met Office FAMOUS v6.11 i686-apple-darwin |
Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.4 build 10F569 Wed Sep 8 22:37:07 2010 sh: /usr/bin/atos: No such file or directory 0 0x0048fdcf 1 0x0049054a 2 0x90b451fb 3 0xffffffff 4 0x0138f6b3 5 0x0138e44f 6 0x003f5826 7 0x00450bd8 8 0x004698c0 9 0x000026b6 10 0x000025dd Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd47c edx: 0x90adf0fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4b8 esp: 0xbfffd47c ss: 0x0000001f efl: 0x00000206 eip: 0x90adf0fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12ea000 - 0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libsvml.dylib 0x135d000 - 0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libifcoremt.dylib 0x143d000 - 0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libimf.dylib 0x16aa000 - 0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libirc.dylib 0x90ade000 - 0x90c84fff /usr/lib/libSystem.B.dylib 0x92609000 - 0x9260cfff /usr/lib/system/libmathCommon.A.dylib 0x95f56000 - 0x95f64fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1394, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.4 build 10F569 Wed Sep 8 23:54:34 2010 sh: /usr/bin/atos: No such file or directory 0 0x0048fdcf 1 0x0049054a 2 0x93f6f1fb 3 0xffffffff 4 0x0138f6b3 5 0x0138e44f 6 0x002eeac6 7 0x003f55ea 8 0x00452abd 9 0x004698c0 10 0x000026b6 11 0x000025dd Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3ec edx: 0x93f090fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc428 esp: 0xbfffc3ec ss: 0x0000001f efl: 0x00000206 eip: 0x93f090fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12ea000 - 0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libsvml.dylib 0x135d000 - 0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libifcoremt.dylib 0x143d000 - 0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libimf.dylib 0x16aa000 - 0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libirc.dylib 0x90003000 - 0x90006fff /usr/lib/system/libmathCommon.A.dylib 0x91127000 - 0x91135fff /usr/lib/libz.1.dylib 0x93f08000 - 0x940aefff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1133, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.4 build 10F569 Sat Sep 11 00:30:23 2010 sh: /usr/bin/atos: No such file or directory 0 0x0048fdcf 1 0x0049054a 2 0x93f6f1fb 3 0xffffffff 4 0x0138f6b3 5 0x0138e44f 6 0x003f5826 7 0x00450bd8 8 0x004698c0 9 0x000026b6 10 0x000025dd Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd47c edx: 0x93f090fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4b8 esp: 0xbfffd47c ss: 0x0000001f efl: 0x00000206 eip: 0x93f090fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12ea000 - 0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libsvml.dylib 0x135d000 - 0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libifcoremt.dylib 0x143d000 - 0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libimf.dylib 0x16aa000 - 0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libirc.dylib 0x90003000 - 0x90006fff /usr/lib/system/libmathCommon.A.dylib 0x91127000 - 0x91135fff /usr/lib/libz.1.dylib 0x93f08000 - 0x940aefff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8146, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.4 build 10F569 Sat Sep 11 05:03:07 2010 sh: /usr/bin/atos: No such file or directory 0 0x0048fdcf 1 0x0049054a 2 0x93f6f1fb 3 0xffffffff 4 0x0138f6b3 5 0x0138e44f 6 0x003f5826 7 0x00450bd8 8 0x004698c0 9 0x000026b6 10 0x000025dd Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd47c edx: 0x93f090fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4b8 esp: 0xbfffd47c ss: 0x0000001f efl: 0x00000206 eip: 0x93f090fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12ea000 - 0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libsvml.dylib 0x135d000 - 0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libifcoremt.dylib 0x143d000 - 0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libimf.dylib 0x16aa000 - 0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libirc.dylib 0x90003000 - 0x90006fff /usr/lib/system/libmathCommon.A.dylib 0x91127000 - 0x91135fff /usr/lib/libz.1.dylib 0x93f08000 - 0x940aefff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12824, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.4 build 10F569 Sat Sep 11 08:40:08 2010 sh: /usr/bin/atos: No such file or directory 0 0x0048fdcf 1 0x0049054a 2 0x93f6f1fb 3 0xffffffff 4 0x0138f6b3 5 0x0138e44f 6 0x003f5826 7 0x00450bd8 8 0x004698c0 9 0x000026b6 10 0x000025dd Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd47c edx: 0x93f090fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4b8 esp: 0xbfffd47c ss: 0x0000001f efl: 0x00000206 eip: 0x93f090fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12ea000 - 0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libsvml.dylib 0x135d000 - 0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libifcoremt.dylib 0x143d000 - 0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libimf.dylib 0x16aa000 - 0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libirc.dylib 0x90003000 - 0x90006fff /usr/lib/system/libmathCommon.A.dylib 0x91127000 - 0x91135fff /usr/lib/libz.1.dylib 0x93f08000 - 0x940aefff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16446, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.4 build 10F569 Sun Sep 12 11:17:12 2010 sh: /usr/bin/atos: No such file or directory 0 0x0048fdcf 1 0x0049054a 2 0x93f6f1fb 3 0xffffffff 4 0x0138f6b3 5 0x0138e44f 6 0x003f5826 7 0x00450bd8 8 0x004698c0 9 0x000026b6 10 0x000025dd Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd47c edx: 0x93f090fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd4b8 esp: 0xbfffd47c ss: 0x0000001f efl: 0x00000206 eip: 0x93f090fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12ea000 - 0x1305fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libsvml.dylib 0x135d000 - 0x13f4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libifcoremt.dylib 0x143d000 - 0x1597fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libimf.dylib 0x16aa000 - 0x16d8fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_v2vd_1999_200_006689487/lib/libirc.dylib 0x90003000 - 0x90006fff /usr/lib/system/libmathCommon.A.dylib 0x91127000 - 0x91135fff /usr/lib/libz.1.dylib 0x93f08000 - 0x940aefff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37542, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (37542): called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
12 Sep 2010 09:20:42 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,516,346 | 211,584 | 0.1395 |
12 Sep 2010 08:53:44 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,506,986 | 210,256 | 0.1395 |
12 Sep 2010 08:32:41 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,497,626 | 208,946 | 0.1395 |
12 Sep 2010 08:01:23 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,488,266 | 207,637 | 0.1395 |
12 Sep 2010 07:35:14 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,478,906 | 206,350 | 0.1395 |
12 Sep 2010 06:26:16 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,469,546 | 205,035 | 0.1395 |
12 Sep 2010 06:05:08 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,460,186 | 203,724 | 0.1395 |
12 Sep 2010 05:39:04 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,450,826 | 202,422 | 0.1395 |
12 Sep 2010 04:18:52 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,441,466 | 201,109 | 0.1395 |
12 Sep 2010 03:57:44 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,432,106 | 199,791 | 0.1395 |
12 Sep 2010 03:31:20 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,422,746 | 198,473 | 0.1395 |
12 Sep 2010 03:10:00 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,413,386 | 197,176 | 0.1395 |
12 Sep 2010 02:43:15 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,404,026 | 195,877 | 0.1395 |
12 Sep 2010 02:22:10 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,394,666 | 194,582 | 0.1395 |
12 Sep 2010 02:00:02 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,385,306 | 193,286 | 0.1395 |
12 Sep 2010 00:05:13 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,375,946 | 191,990 | 0.1395 |
11 Sep 2010 23:51:40 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,366,586 | 190,685 | 0.1395 |
11 Sep 2010 22:20:08 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,357,226 | 189,358 | 0.1395 |
11 Sep 2010 21:58:03 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,347,866 | 188,058 | 0.1395 |
11 Sep 2010 21:32:02 | 1072552 | 11707205 | famous_v2vd_1999_200_006689487_1 | 1,338,506 | 186,738 | 0.1395 |
©2024 climateprediction.net