climateprediction.net home page
Task 11430260

Task 11430260

Name famous_u2co_1599_200_006635891_0
Workunit 6839263
Created 10 Jun 2010, 11:27:05 UTC
Sent 18 Jul 2010, 18:12:45 UTC
Report deadline 18 Oct 2010, 1:39:56 UTC
Received 8 Sep 2010, 15:33:14 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 978826
Run time 6 days 15 hours 33 min 16 sec
CPU time 5 days 13 hours 56 min 24 sec
Validate state Invalid
Credit 1,914.75
Device peak FLOPS 1.23 GFLOPS
Application version UK Met Office FAMOUS v6.11
windows_intelx86
Stderr
<core_client_version>6.10.18</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5860, selfPID=5860, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5736, selfPID=5736, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5972, selfPID=5972, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4892, selfPID=4892, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5484, selfPID=5484, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5896, selfPID=5896, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4936, selfPID=4936, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5780, selfPID=5780, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2840, selfPID=2840, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2876, selfPID=2876, iMonCtr=1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=900, selfPID=900, iMonCtr=1

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5216, selfPID=5216, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3888, selfPID=3888, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4676, selfPID=4676, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5924, selfPID=5924, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=552, selfPID=552, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3280, selfPID=3280, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2160, selfPID=2160, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3304, selfPID=3304, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5708, selfPID=5708, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1520, selfPID=1520, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5776, selfPID=5776, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5812, selfPID=5812, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=336, selfPID=336, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4276, selfPID=4276, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5952, selfPID=5952, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3948, selfPID=3948, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1788, selfPID=1788, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6000, selfPID=6000, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1480, selfPID=1480, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2120, selfPID=2120, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6076, selfPID=6076, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4488, selfPID=4488, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4048, selfPID=4048, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1272, selfPID=1272, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6020, selfPID=6020, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1404, selfPID=1404, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1572, selfPID=1572, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5180, selfPID=5180, iMonCtr=1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5664, selfPID=5664, iMonCtr=1

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: Result too large
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
20:41:58 (6044): Can't acquire lockfile (32) - waiting 35s
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3520, selfPID=3520, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2164, selfPID=2164, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4656, selfPID=4656, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3860, selfPID=3860, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3196, selfPID=3196, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5544, selfPID=5544, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5852, selfPID=5852, iMonCtr=1
07:39:26 (4664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:39:27 (4664): No heartbeat from core client for 30 sec - exiting
07:39:28 (4664): No heartbeat from core client for 30 sec - exiting
07:39:29 (4664): No heartbeat from core client for 30 sec - exiting
07:39:30 (4664): No heartbeat from core client for 30 sec - exiting
07:39:31 (4664): No heartbeat from core client for 30 sec - exiting
07:39:33 (4664): No heartbeat from core client for 30 sec - exiting
07:39:34 (4664): No heartbeat from core client for 30 sec - exiting
07:39:35 (4664): No heartbeat from core client for 30 sec - exiting
07:39:36 (4664): No heartbeat from core client for 30 sec - exiting
07:39:37 (4664): No heartbeat from core client for 30 sec - exiting
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5204, selfPID=5204, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6088, selfPID=6088, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5720, selfPID=5720, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1112, selfPID=1112, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5088, selfPID=5088, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4980, selfPID=4980, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5744, selfPID=5744, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2592, selfPID=2592, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6084, selfPID=6084, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4716, selfPID=4716, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2020, selfPID=2020, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5124, selfPID=5124, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3608, selfPID=3608, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4260, selfPID=4260, iMonCtr=1
06:49:14 (6036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:49:15 (6036): No heartbeat from core client for 30 sec - exiting
06:49:16 (6036): No heartbeat from core client for 30 sec - exiting
06:49:17 (6036): No heartbeat from core client for 30 sec - exiting
06:49:18 (6036): No heartbeat from core client for 30 sec - exiting
06:49:19 (6036): No heartbeat from core client for 30 sec - exiting
06:49:20 (6036): No heartbeat from core client for 30 sec - exiting
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4456, selfPID=4456, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5748, selfPID=5748, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6000, selfPID=6000, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1452, selfPID=1452, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5152, selfPID=5152, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5484, selfPID=5484, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3736, selfPID=3736, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2016, selfPID=2016, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5516, selfPID=5516, iMonCtr=1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5348, selfPID=5348, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2368, selfPID=2368, iMonCtr=1
06:40:37 (2332): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:40:38 (2332): No heartbeat from core client for 30 sec - exiting
06:40:39 (2332): No heartbeat from core client for 30 sec - exiting
06:40:40 (2332): No heartbeat from core client for 30 sec - exiting
06:40:41 (2332): No heartbeat from core client for 30 sec - exiting
06:40:43 (2332): No heartbeat from core client for 30 sec - exiting
06:40:45 (2332): No heartbeat from core client for 30 sec - exiting
06:40:46 (2332): No heartbeat from core client for 30 sec - exiting
06:40:47 (5996): Can't acquire lockfile (32) - waiting 35s
06:40:47 (2332): No heartbeat from core client for 30 sec - exiting
06:40:48 (2332): No heartbeat from core client for 30 sec - exiting
06:40:49 (2332): No heartbeat from core client for 30 sec - exiting
06:40:50 (2332): No heartbeat from core client for 30 sec - exiting
06:40:51 (2332): No heartbeat from core client for 30 sec - exiting
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2764, selfPID=2764, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4916, selfPID=4916, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5972, selfPID=5972, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4720, selfPID=4720, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2568, selfPID=2568, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6116, selfPID=6116, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4372, selfPID=4372, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6812, selfPID=6812, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7112, selfPID=7112, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
No Process Handle
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3272, selfPID=3272, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6548, selfPID=6548, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_u2co_1599_200_006635891\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_u2co_1599_200_006635891\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_u2co_1599_200_006635891\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_u2co_1599_200_006635891\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_u2co_1599_200_006635891\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_u2co_1599_200_006635891\dataout\stdout_um.txt
Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6016, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
06:42:36 (6016): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
05 Sep 2010 18:57:59 978826 11430260 famous_u2co_1599_200_006635891_0 580,346 474,828 0.8182
04 Sep 2010 20:17:28 978826 11430260 famous_u2co_1599_200_006635891_0 570,986 467,211 0.8183
04 Sep 2010 18:37:25 978826 11430260 famous_u2co_1599_200_006635891_0 561,626 459,446 0.8181
04 Sep 2010 18:37:25 978826 11430260 famous_u2co_1599_200_006635891_0 552,266 451,786 0.8181
04 Sep 2010 18:37:25 978826 11430260 famous_u2co_1599_200_006635891_0 542,906 444,035 0.8179
31 Aug 2010 02:59:45 978826 11430260 famous_u2co_1599_200_006635891_0 533,546 436,395 0.8179
26 Aug 2010 21:50:23 978826 11430260 famous_u2co_1599_200_006635891_0 524,186 428,813 0.8181
26 Aug 2010 19:36:46 978826 11430260 famous_u2co_1599_200_006635891_0 514,826 421,274 0.8183
26 Aug 2010 17:29:46 978826 11430260 famous_u2co_1599_200_006635891_0 505,466 413,739 0.8185
26 Aug 2010 01:33:47 978826 11430260 famous_u2co_1599_200_006635891_0 496,106 406,204 0.8188
25 Aug 2010 22:22:18 978826 11430260 famous_u2co_1599_200_006635891_0 486,746 398,588 0.8189
25 Aug 2010 20:09:29 978826 11430260 famous_u2co_1599_200_006635891_0 477,386 391,010 0.8191
25 Aug 2010 17:59:50 978826 11430260 famous_u2co_1599_200_006635891_0 468,026 383,319 0.8190
25 Aug 2010 14:20:47 978826 11430260 famous_u2co_1599_200_006635891_0 458,666 375,586 0.8189
24 Aug 2010 20:03:37 978826 11430260 famous_u2co_1599_200_006635891_0 449,306 367,894 0.8188
24 Aug 2010 17:36:51 978826 11430260 famous_u2co_1599_200_006635891_0 439,946 360,288 0.8189
24 Aug 2010 02:44:57 978826 11430260 famous_u2co_1599_200_006635891_0 430,586 352,588 0.8189
20 Aug 2010 00:22:10 978826 11430260 famous_u2co_1599_200_006635891_0 421,226 344,971 0.8190
19 Aug 2010 22:03:29 978826 11430260 famous_u2co_1599_200_006635891_0 411,866 337,249 0.8188
19 Aug 2010 19:29:43 978826 11430260 famous_u2co_1599_200_006635891_0 402,506 329,443 0.8185


©2024 climateprediction.net