climateprediction.net home page
Task 11772754

Task 11772754

Name famous_vczj_1599_200_006702597_0
Workunit 6905850
Created 26 Aug 2010, 16:46:05 UTC
Sent 30 Nov 2010, 9:47:01 UTC
Report deadline 1 Mar 2011, 17:14:12 UTC
Received 31 Dec 2010, 9:18:44 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1100176
Run time
CPU time 8 days 1 hours 9 min 40 sec
Validate state Invalid
Credit 4,138.23
Device peak FLOPS 2.60 GFLOPS
Application version UK Met Office FAMOUS v6.11
i686-pc-linux-gnu
Stderr
<core_client_version>6.2.18</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
 (17158): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (887): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
 (887): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  
CPDN Monitor - Quit request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  
Sorry, too many model crashes! :-(
 (11958): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
31 Dec 2010 03:04:47 1100176 11772754 famous_vczj_1599_200_006702597_0 1,254,266 691,429 0.5513
30 Dec 2010 23:32:30 1100176 11772754 famous_vczj_1599_200_006702597_0 1,244,906 686,258 0.5513
30 Dec 2010 22:03:37 1100176 11772754 famous_vczj_1599_200_006702597_0 1,235,546 681,108 0.5513
30 Dec 2010 18:33:18 1100176 11772754 famous_vczj_1599_200_006702597_0 1,226,186 675,952 0.5513
30 Dec 2010 14:54:01 1100176 11772754 famous_vczj_1599_200_006702597_0 1,216,826 670,790 0.5513
30 Dec 2010 12:21:30 1100176 11772754 famous_vczj_1599_200_006702597_0 1,207,466 665,627 0.5513
30 Dec 2010 09:52:19 1100176 11772754 famous_vczj_1599_200_006702597_0 1,198,106 660,465 0.5513
30 Dec 2010 06:21:42 1100176 11772754 famous_vczj_1599_200_006702597_0 1,188,746 655,312 0.5513
30 Dec 2010 03:53:30 1100176 11772754 famous_vczj_1599_200_006702597_0 1,179,386 650,151 0.5513
30 Dec 2010 00:17:55 1100176 11772754 famous_vczj_1599_200_006702597_0 1,170,026 644,988 0.5513
29 Dec 2010 21:43:22 1100176 11772754 famous_vczj_1599_200_006702597_0 1,160,666 639,821 0.5513
29 Dec 2010 18:40:24 1100176 11772754 famous_vczj_1599_200_006702597_0 1,151,306 634,658 0.5513
29 Dec 2010 15:42:39 1100176 11772754 famous_vczj_1599_200_006702597_0 1,141,946 629,498 0.5513
29 Dec 2010 12:08:29 1100176 11772754 famous_vczj_1599_200_006702597_0 1,132,586 624,341 0.5513
29 Dec 2010 09:32:02 1100176 11772754 famous_vczj_1599_200_006702597_0 1,123,226 619,186 0.5513
29 Dec 2010 03:34:36 1100176 11772754 famous_vczj_1599_200_006702597_0 1,104,506 608,882 0.5513
29 Dec 2010 01:04:07 1100176 11772754 famous_vczj_1599_200_006702597_0 1,095,146 603,761 0.5513
28 Dec 2010 21:22:46 1100176 11772754 famous_vczj_1599_200_006702597_0 1,085,786 598,609 0.5513
28 Dec 2010 18:48:20 1100176 11772754 famous_vczj_1599_200_006702597_0 1,076,426 593,451 0.5513
28 Dec 2010 18:31:25 1100176 11772754 famous_vczj_1599_200_006702597_0 1,067,066 588,300 0.5513


©2024 climateprediction.net