climateprediction.net home page
Task 13113146

Task 13113146

Name hadcm3n_ygt0_1900_40_007354638_0
Workunit 7552068
Created 6 Jul 2011, 14:34:54 UTC
Sent 11 Jul 2011, 0:21:02 UTC
Report deadline 10 Oct 2011, 7:48:13 UTC
Received 21 Aug 2011, 1:55:19 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1094815
Run time 8 days 9 hours 14 min 33 sec
CPU time 8 days 4 hours 51 min 22 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 1.45 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:17:37 (1692): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:49 (3416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:19:32 (3422): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:21:14 (3428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:22:57 (3435): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:24:40 (3443): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:27:38 (3449): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:33:32 (3455): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:37:06 (3461): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:16:52 (3469): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:41:02 (3514): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:15:00 (3585): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:53:22 (3659): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:59:22 (3829): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:01:04 (4058): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:02:47 (4066): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:04:30 (4213): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:06:13 (4236): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:07:55 (4242): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:09:38 (4248): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:11:21 (4264): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:13:04 (4270): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:14:47 (4278): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:16:30 (4284): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:18:13 (4290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:19:57 (4300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:21:38 (4306): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:23:21 (4313): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:25:04 (4319): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:26:47 (4325): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:27:46 (4333): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:29:08 (4339): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:30:51 (4345): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:32:33 (4349): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:34:16 (4353): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:35:58 (4357): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:37:41 (4361): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:39:23 (4365): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:41:06 (4381): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:41:49 (4385): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:43:11 (4389): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:44:53 (4393): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:46:36 (4400): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:48:18 (4404): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:50:01 (4408): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:51:43 (4413): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:53:26 (4420): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:55:09 (4424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:56:51 (4428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:58:34 (4435): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:00:16 (4439): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:01:59 (4443): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:03:41 (4447): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:05:24 (4451): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:07:07 (4457): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:08:49 (4461): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:10:32 (4465): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:12:14 (4479): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:13:57 (4483): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:15:39 (4490): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:17:22 (4494): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:19:04 (4503): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:20:47 (4509): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:22:29 (4517): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:24:12 (4521): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:25:55 (4527): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:27:37 (4531): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:28:25 (4535): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:29:22 (4539): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:30:09 (4543): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:31:06 (4547): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:31:54 (4553): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:32:51 (4557): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:33:38 (4561): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:34:36 (4565): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:35:23 (4569): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:36:20 (4573): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:37:08 (4577): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:38:05 (4581): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:38:52 (4588): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:39:50 (4601): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:40:37 (4605): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:41:34 (4609): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:42:21 (4615): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:43:19 (4619): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:44:06 (4623): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:45:03 (4632): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:45:51 (4636): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:46:48 (4640): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:47:35 (4644): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:48:33 (4648): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:49:20 (4652): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:50:17 (4656): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:51:05 (4664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:52:02 (4668): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:52:49 (4672): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:53:47 (4676): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:54:34 (4680): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:55:31 (4684): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:56:18 (4688): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:57:16 (4692): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:58:03 (4698): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:59:00 (4702): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:59:48 (4706): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:00:45 (4710): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:01:32 (4714): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:02:29 (4719): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:03:17 (4726): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:04:14 (4730): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08978768 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xa8b591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xa8cde8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xa8fecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0x1cb741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0x1cb79d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xa36bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
00110000-001f9000 r-xp 00000000 08:05 791731     /usr/lib/libstdc++.so.6.0.13
001f9000-001fa000 ---p 000e9000 08:05 791731     /usr/lib/libstdc++.so.6.0.13
001fa000-001fe000 r--p 000e9000 08:05 791731     /usr/lib/libstdc++.so.6.0.13
001fe000-001ff000 rw-p 000ed000 08:05 791731     /usr/lib/libstdc++.so.6.0.13
001ff000-00206000 rw-p 00000000 00:00 0 
00237000-00252000 r-xp 00000000 08:05 131908     /lib/ld-2.11.1.so
00252000-00253000 r--p 0001a000 08:05 131908     /lib/ld-2.11.1.so
00253000-00254000 rw-p 0001b000 08:05 131908     /lib/ld-2.11.1.so
0032c000-00350000 r-xp 00000000 08:05 148235     /lib/tls/i686/cmov/libm-2.11.1.so
00350000-00351000 r--p 00023000 08:05 148235     /lib/tls/i686/cmov/libm-2.11.1.so
00351000-00352000 rw-p 00024000 08:05 148235     /lib/tls/i686/cmov/libm-2.11.1.so
0043c000-0043e000 r-xp 00000000 08:05 148234     /lib/tls/i686/cmov/libdl-2.11.1.so
0043e000-0043f000 r--p 00001000 08:05 148234     /lib/tls/i686/cmov/libdl-2.11.1.so
0043f000-00440000 rw-p 00002000 08:05 148234     /lib/tls/i686/cmov/libdl-2.11.1.so
006e8000-00705000 r-xp 00000000 08:05 131659     /lib/libgcc_s.so.1
00705000-00706000 r--p 0001c000 08:05 131659     /lib/libgcc_s.so.1
00706000-00707000 rw-p 0001d000 08:05 131659     /lib/libgcc_s.so.1
00727000-0073c000 r-xp 00000000 08:05 148248     /lib/tls/i686/cmov/libpthread-2.11.1.so
0073c000-0073d000 r--p 00014000 08:05 148248     /lib/tls/i686/cmov/libpthread-2.11.1.so
0073d000-0073e000 rw-p 00015000 08:05 148248     /lib/tls/i686/cmov/libpthread-2.11.1.so
0073e000-00740000 rw-p 00000000 00:00 0 
00a20000-00b73000 r-xp 00000000 08:05 148217     /lib/tls/i686/cmov/libc-2.11.1.so
00b73000-00b74000 ---p 00153000 08:05 148217     /lib/tls/i686/cmov/libc-2.11.1.so
00b74000-00b76000 r--p 00153000 08:05 148217     /lib/tls/i686/cmov/libc-2.11.1.so
00b76000-00b77000 rw-p 00155000 08:05 148217     /lib/tls/i686/cmov/libc-2.11.1.so
00b77000-00b7a000 rw-p 00000000 00:00 0 
00c6e000-00c6f000 r-xp 00000000 00:00 0          [vdso]
08048000-080e3000 r-xp 00000000 08:05 525434     /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:05 525434     /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
08923000-08989000 rw-p 00000000 00:00 0          [heap]
b7200000-b7221000 rw-p 00000000 00:00 0 
b7221000-b7300000 ---p 00000000 00:00 0 
b7391000-b780c000 rw-s 00000000 08:05 394783     /var/lib/boinc-client/slots/1/136795
b780c000-b780f000 rw-p 00000000 00:00 0 
b7821000-b7822000 rw-p 00000000 00:00 0 
b7822000-b7823000 ---p 00000000 00:00 0 
b7823000-b7826000 rw-p 00000000 00:00 0 
b7826000-b7828000 rw-s 00000000 08:05 405891     /var/lib/boinc-client/slots/1/boinc_mmap_file
b7828000-b782a000 rw-p 00000000 00:00 0 
bf9f4000-bfa64000 rw-p 00000000 00:00 0          [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xc6e400]
[0xc6e422]
/lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0xa4a651]
/lib/tls/i686/cmov/libc.so.6(abort+0x182)[0xa4da82]
/lib/tls/i686/cmov/libc.so.6(+0x6149d)[0xa8149d]
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xa8b591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xa8cde8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xa8fecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0x1cb741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0x1cb79d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xa36bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
20 Aug 2011 22:00:40 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 259,200 708,675 2.7341
20 Aug 2011 02:17:59 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 233,280 638,352 2.7364
18 Aug 2011 21:30:53 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 207,360 565,810 2.7286
17 Aug 2011 22:22:35 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 181,440 496,860 2.7384
17 Aug 2011 03:59:08 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 155,520 429,807 2.7637
16 Aug 2011 08:59:41 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 129,600 362,845 2.7997
15 Aug 2011 10:45:33 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 103,680 292,914 2.8252
06 Aug 2011 07:07:00 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 77,760 220,827 2.8399
26 Jul 2011 20:57:25 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 51,840 147,079 2.8372
25 Jul 2011 13:34:57 1094815 13113146 hadcm3n_ygt0_1900_40_007354638_0 25,920 74,083 2.8581


©2024 cpdn.org