climateprediction.net home page
Task 11989345

Task 11989345

Name famous_vjil_1599_200_006711059_6
Workunit 6914312
Created 9 Nov 2010, 17:43:47 UTC
Sent 9 Nov 2010, 17:45:38 UTC
Report deadline 9 Feb 2011, 1:12:49 UTC
Received 12 Nov 2010, 0:46:41 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1093122
Run time 1 days 0 hours 44 min 46 sec
CPU time 19 hours 55 min 58 sec
Validate state Invalid
Credit 555.96
Device peak FLOPS 2.27 GFLOPS
Application version UK Met Office FAMOUS v6.11
windows_intelx86
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
16:58:13 (8044): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:58:25 (8044): No heartbeat from core client for 30 sec - exiting
16:58:26 (8044): No heartbeat from core client for 30 sec - exiting
16:58:43 (8044): No heartbeat from core client for 30 sec - exiting
16:58:45 (8044): No heartbeat from core client for 30 sec - exiting
16:58:46 (8044): No heartbeat from core client for 30 sec - exiting
16:58:48 (8044): No heartbeat from core client for 30 sec - exiting
16:58:49 (8044): No heartbeat from core client for 30 sec - exiting
16:58:50 (8044): No heartbeat from core client for 30 sec - exiting
16:58:51 (8044): No heartbeat from core client for 30 sec - exiting
16:58:54 (8044): No heartbeat from core client for 30 sec - exiting
16:58:56 (8044): No heartbeat from core client for 30 sec - exiting
16:59:07 (8044): No heartbeat from core client for 30 sec - exiting
17:15:30 (1184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:15:36 (1184): No heartbeat from core client for 30 sec - exiting

Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO                                                                                                                                                                                                      tmp/pipe_dummy                                                                  
17:23:32 (3532): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:23:37 (3532): No heartbeat from core client for 30 sec - exiting
17:27:14 (668): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:27:28 (668): No heartbeat from core client for 30 sec - exiting
17:27:33 (668): No heartbeat from core client for 30 sec - exiting
17:27:37 (668): No heartbeat from core client for 30 sec - exiting

Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO                                                                                                                                                                                                      tmp/pipe_dummy                                                                  
18:49:16 (2176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:49:21 (2176): No heartbeat from core client for 30 sec - exiting
18:49:25 (2176): No heartbeat from core client for 30 sec - exiting
18:49:27 (2176): No heartbeat from core client for 30 sec - exiting
18:49:32 (2176): No heartbeat from core client for 30 sec - exiting
18:49:33 (2176): No heartbeat from core client for 30 sec - exiting
18:49:36 (2176): No heartbeat from core client for 30 sec - exiting
18:49:37 (2176): No heartbeat from core client for 30 sec - exiting
18:49:38 (2176): No heartbeat from core client for 30 sec - exiting
18:49:43 (2176): No heartbeat from core client for 30 sec - exiting
18:49:47 (2176): No heartbeat from core client for 30 sec - exiting
18:49:51 (2176): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
21:30:14 (7268): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:30:17 (7268): No heartbeat from core client for 30 sec - exiting
21:30:19 (7268): No heartbeat from core client for 30 sec - exiting
21:31:08 (7268): No heartbeat from core client for 30 sec - exiting
21:31:14 (7268): No heartbeat from core client for 30 sec - exiting
21:31:16 (7268): No heartbeat from core client for 30 sec - exiting
21:31:21 (7268): No heartbeat from core client for 30 sec - exiting
21:31:23 (7268): No heartbeat from core client for 30 sec - exiting
21:31:25 (7268): No heartbeat from core client for 30 sec - exiting
21:31:29 (7268): No heartbeat from core client for 30 sec - exiting
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6096, iMonCtr=1
Model crash detected, will try to restart...

Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO                                                                                                                                                                                                      tmp/pipe_dummy                                                                  
00:33:48 (6096): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:34:03 (6096): No heartbeat from core client for 30 sec - exiting
00:45:25 (7536): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:45:38 (7536): No heartbeat from core client for 30 sec - exiting
00:45:42 (7536): No heartbeat from core client for 30 sec - exiting
00:46:01 (7536): No heartbeat from core client for 30 sec - exiting
01:11:21 (1052): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:11:33 (1052): No heartbeat from core client for 30 sec - exiting
01:11:36 (1052): No heartbeat from core client for 30 sec - exiting
01:11:37 (1052): No heartbeat from core client for 30 sec - exiting
01:11:47 (1052): No heartbeat from core client for 30 sec - exiting
01:11:56 (1052): No heartbeat from core client for 30 sec - exiting
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1716, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1716, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1716, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1716, iMonCtr=1
Model crash detected, will try to restart...
01:32:35 (1716): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:10:10 (7320): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:02:44 (3524): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:03:01 (3524): No heartbeat from core client for 30 sec - exiting
03:03:04 (3524): No heartbeat from core client for 30 sec - exiting
03:03:05 (3524): No heartbeat from core client for 30 sec - exiting
03:03:06 (3524): No heartbeat from core client for 30 sec - exiting
03:03:12 (3524): No heartbeat from core client for 30 sec - exiting
03:03:19 (3524): No heartbeat from core client for 30 sec - exiting
03:03:20 (3524): No heartbeat from core client for 30 sec - exiting
03:03:22 (3524): No heartbeat from core client for 30 sec - exiting
03:03:25 (3524): No heartbeat from core client for 30 sec - exiting
03:26:02 (6072): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:26:04 (6072): No heartbeat from core client for 30 sec - exiting
03:26:05 (6072): No heartbeat from core client for 30 sec - exiting
03:26:06 (6072): No heartbeat from core client for 30 sec - exiting
03:26:07 (6072): No heartbeat from core client for 30 sec - exiting
03:26:09 (6072): No heartbeat from core client for 30 sec - exiting
03:26:10 (6072): No heartbeat from core client for 30 sec - exiting
03:26:11 (6072): No heartbeat from core client for 30 sec - exiting
03:26:12 (6072): No heartbeat from core client for 30 sec - exiting
03:26:13 (6072): No heartbeat from core client for 30 sec - exiting
03:26:14 (6072): No heartbeat from core client for 30 sec - exiting
03:31:24 (6528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:50:01 (2888): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:50:07 (2888): No heartbeat from core client for 30 sec - exiting
04:50:12 (2888): No heartbeat from core client for 30 sec - exiting
04:50:13 (2888): No heartbeat from core client for 30 sec - exiting
04:50:18 (2888): No heartbeat from core client for 30 sec - exiting
04:50:23 (2888): No heartbeat from core client for 30 sec - exiting
005:01:37 (1184): No heartbeat from core client for 30 sec - exiting
05:01:41 (1184): No heartbeat from core client for 30 sec - exiting
05:01:59 (1184): No heartbeat from core client for 30 sec - exiting
05:02:00 (1184): No heartbeat from core client for 30 sec - exiting
05:02:03 (1184): No heartbeat from core client for 30 sec - exiting
05:02:04 (1184): No heartbeat from core client for 30 sec - exiting
05:02:07 (1184): No heartbeat from core client for 30 sec - exiting
05:02:18 (1184): No heartbeat from core client for 30 sec - exiting
05:02:20 (1184): No heartbeat from core client for 30 sec - exiting
05:02:21 (1184): No heartbeat from core client for 30 sec - exiting
05:02:23 (1184): No heartbeat from core client for 30 sec - exiting
05:02:24 (1184): No heartbeat from core client for 30 sec - exiting
05:02:25 (1184): No heartbeat from core client for 30 sec - exiting
05:02:26 (1184): No heartbeat from core client for 30 sec - exiting
05:02:28 (1184): No heartbeat from core client for 30 sec - exiting
05:02:29 (1184): No heartbeat from core client for 30 sec - exiting
05:02:30 (1184): No heartbeat from core client for 30 sec - exiting
05:02:31 (1184): No heartbeat from core client for 30 sec - exiting
05:02:32 (1184): No heartbeat from core client for 30 sec - exiting
05:02:33 (1184): No heartbeat from core client for 30 sec - exiting
05:02:44 (1184): No heartbeat from core client for 30 sec - exiting
05:02:49 (1184): No heartbeat from core client for 30 sec - exiting
05:02:52 (1184): No heartbeat from core client for 30 sec - exiting
05:02:55 (1184): No heartbeat from core client for 30 sec - exiting
05:02:56 (1184): No heartbeat from core client for 30 sec - exiting
05:02:57 (1184): No heartbeat from core client for 30 sec - exiting
05:02:58 (1184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
05:57:13 (6232): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:57:30 (6232): No heartbeat from core client for 30 sec - exiting
05:57:32 (6232): No heartbeat from core client for 30 sec - exiting
05:57:33 (6232): No heartbeat from core client for 30 sec - exiting
05:57:38 (6232): No heartbeat from core client for 30 sec - exiting
05:57:41 (6232): No heartbeat from core client for 30 sec - exiting
06:27:46 (8120): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:27:48 (8120): No heartbeat from core client for 30 sec - exiting
06:27:49 (8120): No heartbeat from core client for 30 sec - exiting
07:02:01 (6464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:37:26 (7836): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:37:47 (7836): No heartbeat from core client for 30 sec - exiting
07:37:54 (7836): No heartbeat from core client for 30 sec - exiting
07:37:58 (7836): No heartbeat from core client for 30 sec - exiting
07:38:02 (7836): No heartbeat from core client for 30 sec - exiting
07:48:10 (7908): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:48:36 (7908): No heartbeat from core client for 30 sec - exiting
07:48:40 (7908): No heartbeat from core client for 30 sec - exiting
07:48:44 (7908): No heartbeat from core client for 30 sec - exiting
10:58:26 (6880): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:10:53 (5780): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:11:50 (5780): No heartbeat from core client for 30 sec - exiting
11:12:29 (5780): No heartbeat from core client for 30 sec - exiting
11:45:22 (3060): No heartbeat from core client for 30 sec - exiting
11:45:34 (3060): No heartbeat from core client for 30 sec - exiting
11:45:35 (3060): No heartbeat from core client for 30 sec - exiting
11:45:36 (3060): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:45:39 (3060): No heartbeat from core client for 30 sec - exiting
11:45:40 (3060): No heartbeat from core client for 30 sec - exiting
11:45:45 (3060): No heartbeat from core client for 30 sec - exiting
11:45:46 (3060): No heartbeat from core client for 30 sec - exiting
11:56:24 (2888): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:10:57 (7416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:11:02 (7416): No heartbeat from core client for 30 sec - exiting
13:11:06 (7416): No heartbeat from core client for 30 sec - exiting
13:11:07 (7416): No heartbeat from core client for 30 sec - exiting
13:11:09 (7416): No heartbeat from core client for 30 sec - exiting
13:11:10 (7416): No heartbeat from core client for 30 sec - exiting
13:11:13 (7416): No heartbeat from core client for 30 sec - exiting
13:49:15 (3096): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:49:29 (3096): No heartbeat from core client for 30 sec - exiting
13:49:34 (3096): No heartbeat from core client for 30 sec - exiting
14:33:32 (6300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:36:17 (6300): No heartbeat from core client for 30 sec - exiting
15:02:15 (544): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:06:34 (4004): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:07:22 (4004): No heartbeat from core client for 30 sec - exiting
15:07:25 (4004): No heartbeat from core client for 30 sec - exiting
15:07:37 (4004): No heartbeat from core client for 30 sec - exiting
15:07:40 (4004): No heartbeat from core client for 30 sec - exiting
15:07:44 (4004): No heartbeat from core client for 30 sec - exiting
15:07:46 (4004): No heartbeat from core client for 30 sec - exiting
16:52:15 (7988): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:53:38 (7988): No heartbeat from core client for 30 sec - exiting
16:53:42 (7988): No heartbeat from core client for 30 sec - exiting
16:53:47 (7988): No heartbeat from core client for 30 sec - exiting
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3312, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3312, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3312, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (47): write to READONLY file, unit 6, file C:\ProgramData\BOINC\projects\climateprediction.net\famous_vjil_1599_200_006711059\dataout\stdout_um.txt

Image              PC        Routine            Line        Source             
famous_um_6.11_wi  008846D2  Unknown               Unknown  Unknown

Stack trace terminated abnormally.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3312, iMonCtr=1
Model crash detected, will try to restart...

Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO                                                                                                                                                                                                      tmp/pipe_dummy                                                                  
18:50:21 (3312): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:51:08 (3312): No heartbeat from core client for 30 sec - exiting
18:51:12 (3312): No heartbeat from core client for 30 sec - exiting
18:51:13 (3312): No heartbeat from core client for 30 sec - exiting
18:51:20 (3312): No heartbeat from core client for 30 sec - exiting
19:57:49 (6164): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:58:19 (6164): No heartbeat from core client for 30 sec - exiting
19:58:20 (6164): No heartbeat from core client for 30 sec - exiting
19:58:22 (6164): No heartbeat from core client for 30 sec - exiting
19:58:24 (6164): No heartbeat from core client for 30 sec - exiting
19:58:32 (6164): No heartbeat from core client for 30 sec - exiting
20:24:49 (5220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:25:07 (5220): No heartbeat from core client for 30 sec - exiting
20:25:13 (5220): No heartbeat from core client for 30 sec - exiting
20:25:18 (5220): No heartbeat from core client for 30 sec - exiting
20:25:20 (5220): No heartbeat from core client for 30 sec - exiting
20:25:21 (5220): No heartbeat from core client for 30 sec - exiting
20:25:24 (5220): No heartbeat from core client for 30 sec - exiting
20:25:31 (5220): No heartbeat from core client for 30 sec - exiting
20:25:32 (5220): No heartbeat from core client for 30 sec - exiting
20:45:31 (3120): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:45:51 (3120): No heartbeat from core client for 30 sec - exiting
20:57:37 (6912): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:57:48 (6912): No heartbeat from core client for 30 sec - exiting
20:57:55 (6912): No heartbeat from core client for 30 sec - exiting
20:57:58 (6912): No heartbeat from core client for 30 sec - exiting
20:57:59 (6912): No heartbeat from core client for 30 sec - exiting
20:58:03 (6912): No heartbeat from core client for 30 sec - exiting
20:58:06 (6912): No heartbeat from core client for 30 sec - exiting
20:58:07 (6912): No heartbeat from core client for 30 sec - exiting
20:58:08 (6912): No heartbeat from core client for 30 sec - exiting
20:58:23 (6912): No heartbeat from core client for 30 sec - exiting

Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO                                                                                                                                                                                           tmp/pipe_dummy                                                                  

Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO                                                                                                                                                                                           tmp/pipe_dummy                                                                  

Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO                                                                                                                                                                                           tmp/pipe_dummy                                                                  

Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO                                                                                                                                                                                           tmp/pipe_dummy                                                                  

Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO                                                                                                                                                                                           tmp/pipe_dummy                                                                  

Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO                                                                                                                                                                                           tmp/pipe_dummy                                                                  
Sorry, too many model crashes! :-(
21:00:51 (4624): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
12 Nov 2010 00:49:23 1093122 11989345 famous_vjil_1599_200_006711059_6 168,506 71,604 0.4249
12 Nov 2010 00:49:23 1093122 11989345 famous_vjil_1599_200_006711059_6 159,146 67,695 0.4254
12 Nov 2010 00:49:23 1093122 11989345 famous_vjil_1599_200_006711059_6 149,786 63,798 0.4259
12 Nov 2010 00:49:23 1093122 11989345 famous_vjil_1599_200_006711059_6 140,426 59,872 0.4264
12 Nov 2010 00:49:23 1093122 11989345 famous_vjil_1599_200_006711059_6 131,066 55,962 0.4270
12 Nov 2010 00:49:23 1093122 11989345 famous_vjil_1599_200_006711059_6 121,706 52,049 0.4277
11 Nov 2010 10:04:57 1093122 11989345 famous_vjil_1599_200_006711059_6 112,346 48,103 0.4282
11 Nov 2010 08:45:25 1093122 11989345 famous_vjil_1599_200_006711059_6 102,986 44,085 0.4281
11 Nov 2010 07:45:24 1093122 11989345 famous_vjil_1599_200_006711059_6 93,626 40,116 0.4285
11 Nov 2010 07:45:24 1093122 11989345 famous_vjil_1599_200_006711059_6 84,266 36,082 0.4282
11 Nov 2010 07:45:24 1093122 11989345 famous_vjil_1599_200_006711059_6 74,906 32,201 0.4299
11 Nov 2010 07:45:24 1093122 11989345 famous_vjil_1599_200_006711059_6 65,546 28,084 0.4285
10 Nov 2010 23:32:47 1093122 11989345 famous_vjil_1599_200_006711059_6 56,186 24,063 0.4283
10 Nov 2010 22:12:48 1093122 11989345 famous_vjil_1599_200_006711059_6 46,826 20,091 0.4291
10 Nov 2010 20:46:02 1093122 11989345 famous_vjil_1599_200_006711059_6 37,466 16,055 0.4285
10 Nov 2010 19:23:19 1093122 11989345 famous_vjil_1599_200_006711059_6 28,106 12,057 0.4290
10 Nov 2010 17:59:04 1093122 11989345 famous_vjil_1599_200_006711059_6 18,746 8,049 0.4294
10 Nov 2010 16:11:09 1093122 11989345 famous_vjil_1599_200_006711059_6 9,386 4,065 0.4331


©2024 climateprediction.net