climateprediction.net home page
Task 11087844

Task 11087844

Name hadsm3dhet2_jukh_006602611_5
Workunit 6805984
Created 15 Mar 2010, 12:10:39 UTC
Sent 8 Jun 2010, 15:16:28 UTC
Report deadline 21 May 2011, 20:36:28 UTC
Received 18 Jun 2010, 5:25:00 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1074243
Run time 8 days 7 hours 49 min 1 sec
CPU time 6 days 14 hours 25 min 46 sec
Validate state Invalid
Credit 4,962.17
Device peak FLOPS 3.02 GFLOPS
Application version UK Met Office HadSM3 Slab Model v6.08
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.56</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
MainError:	11:47:38 AM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8337ab8]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x817048d]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250464]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x814d652]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x80bf4ed]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c8b1e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86fd1e3]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2032, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833ce23]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833b9db]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250189]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8522c92]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x830104e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86de635]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f77ec]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2032, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833ce23]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833b9db]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250189]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8522c92]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x830104e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86de635]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f77ec]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2032, iMonCtr=1
Model crash detected, will try to restart...
CPDN process is not running, exiting, bRetVal = 1, checkPID=2403, selfPID=2403, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.
MainError:	02:16:02 PM	No files match the supplied pattern.

Model crashed: 7R
SIGSEGV: segmentation violation
Stack trace (14 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833ce23]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833b9db]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250189]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8522c92]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x830104e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86de635]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f77ec]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2417, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833ce23]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833b9db]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250189]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8522c92]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x830104e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86de635]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f77ec]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2417, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833ce23]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833b9db]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250189]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8522c92]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x830104e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86de635]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f77ec]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2417, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833b838]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250189]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8522c92]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x830104e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86de635]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f77ec]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2417, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x876060b]
[0x55573400]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8496338]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833ce23]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x833b9db]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8250189]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x8522c92]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x830104e]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86de635]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86f77ec]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86c0886]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu[0x86feb0f]
/lib32/libc.so.6(__libc_start_main+0xe6)[0x555e5bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadsm3_um_6.08_i686-pc-linux-gnu(pthread_mutex_unlock+0x111)[0x804bb31]

Exiting...
CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2417, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Jun 2010 06:42:47 1074243 11087844 hadsm3dhet2_jukh_006602611_5 21,604 569,164 1.0538
17 Jun 2010 18:10:51 1074243 11087844 hadsm3dhet2_jukh_006602611_5 10,802 558,046 1.0543
17 Jun 2010 14:19:10 1074243 11087844 hadsm3dhet2_jukh_006602611_5 259,248 546,975 1.0549
17 Jun 2010 10:29:01 1074243 11087844 hadsm3dhet2_jukh_006602611_5 248,446 535,716 1.0552
17 Jun 2010 06:33:17 1074243 11087844 hadsm3dhet2_jukh_006602611_5 237,644 524,534 1.0556
17 Jun 2010 02:40:27 1074243 11087844 hadsm3dhet2_jukh_006602611_5 226,842 513,233 1.0558
16 Jun 2010 22:52:46 1074243 11087844 hadsm3dhet2_jukh_006602611_5 216,040 501,825 1.0558
16 Jun 2010 18:54:17 1074243 11087844 hadsm3dhet2_jukh_006602611_5 205,238 490,679 1.0564
16 Jun 2010 14:52:37 1074243 11087844 hadsm3dhet2_jukh_006602611_5 194,436 479,240 1.0563
16 Jun 2010 12:00:59 1074243 11087844 hadsm3dhet2_jukh_006602611_5 183,634 467,973 1.0567
16 Jun 2010 07:07:32 1074243 11087844 hadsm3dhet2_jukh_006602611_5 172,832 456,679 1.0569
16 Jun 2010 03:18:34 1074243 11087844 hadsm3dhet2_jukh_006602611_5 162,030 445,306 1.0570
15 Jun 2010 23:28:55 1074243 11087844 hadsm3dhet2_jukh_006602611_5 151,228 434,203 1.0578
15 Jun 2010 19:28:36 1074243 11087844 hadsm3dhet2_jukh_006602611_5 140,426 422,919 1.0582
15 Jun 2010 15:29:27 1074243 11087844 hadsm3dhet2_jukh_006602611_5 129,624 411,789 1.0589
15 Jun 2010 07:24:49 1074243 11087844 hadsm3dhet2_jukh_006602611_5 118,822 400,437 1.0592
15 Jun 2010 06:58:29 1074243 11087844 hadsm3dhet2_jukh_006602611_5 108,020 389,414 1.0603
15 Jun 2010 06:58:29 1074243 11087844 hadsm3dhet2_jukh_006602611_5 97,218 378,368 1.0614
15 Jun 2010 06:58:29 1074243 11087844 hadsm3dhet2_jukh_006602611_5 86,416 367,365 1.0628
14 Jun 2010 15:03:56 1074243 11087844 hadsm3dhet2_jukh_006602611_5 75,614 356,307 1.0640


©2024 climateprediction.net