climateprediction.net home page
Task 15928260

Task 15928260

Name hadcm3n_4h8q_2020_40_008406722_0
Workunit 8557578
Created 20 Aug 2013, 7:28:24 UTC
Sent 20 Aug 2013, 7:36:44 UTC
Report deadline 19 Nov 2013, 15:03:55 UTC
Received 4 Oct 2013, 10:48:45 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1223320
Run time 14 days 10 hours 34 min 25 sec
CPU time 11 days 12 hours 38 min 25 sec
Validate state Invalid
Credit 7,153.92
Device peak FLOPS 2.86 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>7.0.65</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:16:33 (4065): No heartbeat from core client for 30 sec - exiting
22:16:36 (4065): No heartbeat from core client for 30 sec - exiting
22:16:37 (4065): No heartbeat from core client for 30 sec - exiting
22:16:42 (4065): No heartbeat from core client for 30 sec - exiting
22:16:43 (4065): No heartbeat from core client for 30 sec - exiting
22:16:44 (4065): No heartbeat from core client for 30 sec - exiting
22:16:45 (4065): No heartbeat from core client for 30 sec - exiting
22:16:46 (4065): No heartbeat from core client for 30 sec - exiting
22:16:47 (4065): No heartbeat from core client for 30 sec - exiting
22:16:48 (4065): No heartbeat from core client for 30 sec - exiting
22:16:49 (4065): No heartbeat from core client for 30 sec - exiting
22:16:50 (4065): No heartbeat from core client for 30 sec - exiting
22:16:51 (4065): No heartbeat from core client for 30 sec - exiting
22:16:52 (4065): No heartbeat from core client for 30 sec - exiting
22:16:53 (4065): No heartbeat from core client for 30 sec - exiting
22:16:54 (4065): No heartbeat from core client for 30 sec - exiting
22:16:55 (4065): No heartbeat from core client for 30 sec - exiting
22:16:56 (4065): No heartbeat from core client for 30 sec - exiting
22:16:57 (4065): No heartbeat from core client for 30 sec - exiting
22:16:58 (4065): No heartbeat from core client for 30 sec - exiting
22:16:59 (4065): No heartbeat from core client for 30 sec - exiting
22:17:00 (4065): No heartbeat from core client for 30 sec - exiting
22:17:01 (4065): No heartbeat from core client for 30 sec - exiting
22:17:02 (4065): No heartbeat from core client for 30 sec - exiting
22:17:03 (4065): No heartbeat from core client for 30 sec - exiting
22:17:04 (4065): No heartbeat from core client for 30 sec - exiting
22:17:05 (4065): No heartbeat from core client for 30 sec - exiting
22:17:06 (4065): No heartbeat from core client for 30 sec - exiting
22:17:07 (4065): No heartbeat from core client for 30 sec - exiting
22:17:08 (4065): No heartbeat from core client for 30 sec - exiting
22:17:09 (4065): No heartbeat from core client for 30 sec - exiting
22:17:10 (4065): No heartbeat from core client for 30 sec - exiting
22:17:11 (4065): No heartbeat from core client for 30 sec - exiting
22:17:12 (4065): No heartbeat from core client for 30 sec - exiting
22:17:13 (4065): No heartbeat from core client for 30 sec - exiting
22:17:14 (4065): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:26:29 (3808): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:26:30 (3808): No heartbeat from core client for 30 sec - exiting
22:26:31 (3808): No heartbeat from core client for 30 sec - exiting
22:26:32 (3808): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
23:30:05 (3988): No heartbeat from core client for 30 sec - exiting
23:30:09 (3988): No heartbeat from core client for 30 sec - exiting
23:30:10 (3988): No heartbeat from core client for 30 sec - exiting
23:30:11 (3988): No heartbeat from core client for 30 sec - exiting
23:30:12 (3988): No heartbeat from core client for 30 sec - exiting
23:30:13 (3988): No heartbeat from core client for 30 sec - exiting
23:30:14 (3988): No heartbeat from core client for 30 sec - exiting
23:30:15 (3988): No heartbeat from core client for 30 sec - exiting
23:30:16 (3988): No heartbeat from core client for 30 sec - exiting
23:30:17 (3988): No heartbeat from core client for 30 sec - exiting
23:30:18 (3988): No heartbeat from core client for 30 sec - exiting
23:30:19 (3988): No heartbeat from core client for 30 sec - exiting
23:30:20 (3988): No heartbeat from core client for 30 sec - exiting
23:30:21 (3988): No heartbeat from core client for 30 sec - exiting
23:30:22 (3988): No heartbeat from core client for 30 sec - exiting
23:30:23 (3988): No heartbeat from core client for 30 sec - exiting
23:30:24 (3988): No heartbeat from core client for 30 sec - exiting
23:30:25 (3988): No heartbeat from core client for 30 sec - exiting
23:30:26 (3988): No heartbeat from core client for 30 sec - exiting
23:30:27 (3988): No heartbeat from core client for 30 sec - exiting
23:30:28 (3988): No heartbeat from core client for 30 sec - exiting
23:30:29 (3988): No heartbeat from core client for 30 sec - exiting
23:30:30 (3988): No heartbeat from core client for 30 sec - exiting
23:30:31 (3988): No heartbeat from core client for 30 sec - exiting
23:30:34 (3988): No heartbeat from core client for 30 sec - exiting
23:30:35 (3988): No heartbeat from core client for 30 sec - exiting
23:30:36 (3988): No heartbeat from core client for 30 sec - exiting
23:30:37 (3988): No heartbeat from core client for 30 sec - exiting
23:30:38 (3988): No heartbeat from core client for 30 sec - exiting
23:30:39 (3988): No heartbeat from core client for 30 sec - exiting
23:30:40 (3988): No heartbeat from core client for 30 sec - exiting
23:30:41 (3988): No heartbeat from core client for 30 sec - exiting
23:30:42 (3988): No heartbeat from core client for 30 sec - exiting
23:30:43 (3988): No heartbeat from core client for 30 sec - exiting
23:30:44 (3988): No heartbeat from core client for 30 sec - exiting
23:30:45 (3988): No heartbeat from core client for 30 sec - exiting
23:30:46 (3988): No heartbeat from core client for 30 sec - exiting
23:30:47 (3988): No heartbeat from core client for 30 sec - exiting
23:30:48 (3988): No heartbeat from core client for 30 sec - exiting
23:30:49 (3988): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:30:50 (3988): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
23:30:15 (3910): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:30:17 (3910): No heartbeat from core client for 30 sec - exiting
23:30:18 (3910): No heartbeat from core client for 30 sec - exiting
23:30:19 (3910): No heartbeat from core client for 30 sec - exiting
23:30:20 (3910): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:24:30 (3944): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:15:15 (3912): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
07:39:40 (3733): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:23:11 (3814): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:55:03 (3959): No heartbeat from core client for 30 sec - exiting
08:55:06 (3959): No heartbeat from core client for 30 sec - exiting
08:55:07 (3959): No heartbeat from core client for 30 sec - exiting
08:55:08 (3959): No heartbeat from core client for 30 sec - exiting
08:55:09 (3959): No heartbeat from core client for 30 sec - exiting
08:55:10 (3959): No heartbeat from core client for 30 sec - exiting
08:55:11 (3959): No heartbeat from core client for 30 sec - exiting
08:55:12 (3959): No heartbeat from core client for 30 sec - exiting
08:55:13 (3959): No heartbeat from core client for 30 sec - exiting
08:55:14 (3959): No heartbeat from core client for 30 sec - exiting
08:55:15 (3959): No heartbeat from core client for 30 sec - exiting
08:55:16 (3959): No heartbeat from core client for 30 sec - exiting
08:55:17 (3959): No heartbeat from core client for 30 sec - exiting
08:55:18 (3959): No heartbeat from core client for 30 sec - exiting
08:55:19 (3959): No heartbeat from core client for 30 sec - exiting
08:55:20 (3959): No heartbeat from core client for 30 sec - exiting
08:55:21 (3959): No heartbeat from core client for 30 sec - exiting
08:55:22 (3959): No heartbeat from core client for 30 sec - exiting
08:55:23 (3959): No heartbeat from core client for 30 sec - exiting
08:55:24 (3959): No heartbeat from core client for 30 sec - exiting
08:55:25 (3959): No heartbeat from core client for 30 sec - exiting
08:55:26 (3959): No heartbeat from core client for 30 sec - exiting
08:55:27 (3959): No heartbeat from core client for 30 sec - exiting
08:55:28 (3959): No heartbeat from core client for 30 sec - exiting
08:55:29 (3959): No heartbeat from core client for 30 sec - exiting
08:55:30 (3959): No heartbeat from core client for 30 sec - exiting
08:55:31 (3959): No heartbeat from core client for 30 sec - exiting
08:55:32 (3959): No heartbeat from core client for 30 sec - exiting
08:55:33 (3959): No heartbeat from core client for 30 sec - exiting
08:55:34 (3959): No heartbeat from core client for 30 sec - exiting
08:55:35 (3959): No heartbeat from core client for 30 sec - exiting
08:55:36 (3959): No heartbeat from core client for 30 sec - exiting
08:55:37 (3959): No heartbeat from core client for 30 sec - exiting
08:55:38 (3959): No heartbeat from core client for 30 sec - exiting
08:55:39 (3959): No heartbeat from core client for 30 sec - exiting
08:55:40 (3959): No heartbeat from core client for 30 sec - exiting
08:55:41 (3959): No heartbeat from core client for 30 sec - exiting
08:55:42 (3959): No heartbeat from core client for 30 sec - exiting
08:55:43 (3959): No heartbeat from core client for 30 sec - exiting
08:55:44 (3959): No heartbeat from core client for 30 sec - exiting
08:55:45 (3959): No heartbeat from core client for 30 sec - exiting
08:55:46 (3959): No heartbeat from core client for 30 sec - exiting
08:55:47 (3959): No heartbeat from core client for 30 sec - exiting
08:55:48 (3959): No heartbeat from core client for 30 sec - exiting
08:55:49 (3959): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:14:32 (4096): No heartbeat from core client for 30 sec - exiting
22:14:33 (4096): No heartbeat from core client for 30 sec - exiting
22:14:34 (4096): No heartbeat from core client for 30 sec - exiting
22:14:35 (4096): No heartbeat from core client for 30 sec - exiting
22:14:36 (4096): No heartbeat from core client for 30 sec - exiting
22:14:37 (4096): No heartbeat from core client for 30 sec - exiting
22:14:38 (4096): No heartbeat from core client for 30 sec - exiting
22:14:39 (4096): No heartbeat from core client for 30 sec - exiting
22:14:40 (4096): No heartbeat from core client for 30 sec - exiting
22:14:41 (4096): No heartbeat from core client for 30 sec - exiting
22:14:42 (4096): No heartbeat from core client for 30 sec - exiting
22:14:43 (4096): No heartbeat from core client for 30 sec - exiting
22:14:44 (4096): No heartbeat from core client for 30 sec - exiting
22:14:45 (4096): No heartbeat from core client for 30 sec - exiting
22:14:46 (4096): No heartbeat from core client for 30 sec - exiting
22:14:47 (4096): No heartbeat from core client for 30 sec - exiting
22:14:48 (4096): No heartbeat from core client for 30 sec - exiting
22:14:49 (4096): No heartbeat from core client for 30 sec - exiting
22:14:50 (4096): No heartbeat from core client for 30 sec - exiting
22:14:51 (4096): No heartbeat from core client for 30 sec - exiting
22:14:52 (4096): No heartbeat from core client for 30 sec - exiting
22:14:54 (4096): No heartbeat from core client for 30 sec - exiting
22:14:55 (4096): No heartbeat from core client for 30 sec - exiting
22:14:56 (4096): No heartbeat from core client for 30 sec - exiting
22:14:57 (4096): No heartbeat from core client for 30 sec - exiting
22:14:58 (4096): No heartbeat from core client for 30 sec - exiting
22:14:59 (4096): No heartbeat from core client for 30 sec - exiting
22:15:00 (4096): No heartbeat from core client for 30 sec - exiting
22:15:01 (4096): No heartbeat from core client for 30 sec - exiting
22:15:02 (4096): No heartbeat from core client for 30 sec - exiting
22:15:03 (4096): No heartbeat from core client for 30 sec - exiting
22:15:04 (4096): No heartbeat from core client for 30 sec - exiting
22:15:05 (4096): No heartbeat from core client for 30 sec - exiting
22:15:06 (4096): No heartbeat from core client for 30 sec - exiting
22:15:07 (4096): No heartbeat from core client for 30 sec - exiting
22:15:08 (4096): No heartbeat from core client for 30 sec - exiting
22:15:09 (4096): No heartbeat from core client for 30 sec - exiting
22:15:10 (4096): No heartbeat from core client for 30 sec - exiting
22:15:11 (4096): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
20:30:23 (3848): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:30:11 (5571): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
00:25:41 (12441): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
07:11:34 (7984): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:12:51 (13161): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
07:37:55 (4108): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:10:58 (3907): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7726400]
[0xf7726425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75121df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7515825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74fd4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8761, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf76f9400]
[0xf76f9425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf74e51df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf74e8825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74d04d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8761, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77d4400]
[0xf77d4425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75c01df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75c3825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75ab4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8761, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf774f400]
[0xf774f425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf753b1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf753e825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75264d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8761, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77d0400]
[0xf77d0425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75bc1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75bf825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75a74d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8761, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7771400]
[0xf7771425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf755d1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7560825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75484d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8761, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
04 Oct 2013 00:22:03 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 596,160 981,486 1.6463
03 Oct 2013 10:47:11 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 570,240 939,144 1.6469
02 Oct 2013 18:44:13 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 544,320 895,867 1.6458
02 Oct 2013 05:17:27 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 518,400 852,718 1.6449
01 Oct 2013 12:52:52 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 492,480 809,973 1.6447
30 Sep 2013 20:04:44 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 466,560 767,222 1.6444
30 Sep 2013 04:28:16 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 440,640 724,433 1.6440
29 Sep 2013 14:36:16 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 414,720 681,925 1.6443
28 Sep 2013 21:45:22 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 388,800 638,843 1.6431
28 Sep 2013 04:02:37 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 362,880 596,691 1.6443
27 Sep 2013 06:11:18 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 336,960 554,429 1.6454
26 Sep 2013 14:50:27 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 311,040 511,262 1.6437
25 Sep 2013 16:45:06 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 285,120 467,718 1.6404
25 Sep 2013 09:24:11 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 259,200 424,889 1.6392
24 Sep 2013 03:16:30 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 233,280 381,862 1.6369
23 Sep 2013 10:03:45 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 207,360 339,536 1.6374
23 Sep 2013 10:03:45 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 181,440 296,959 1.6367
21 Sep 2013 17:14:43 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 155,520 255,713 1.6442
20 Sep 2013 22:37:27 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 129,600 213,168 1.6448
20 Sep 2013 05:28:04 1223320 15928260 hadcm3n_4h8q_2020_40_008406722_0 103,680 171,805 1.6571


©2024 climateprediction.net