 |
|
| Name | hadcm3n_u41k_1980_40_008244098_1 |
| Workunit | 8399222 |
| Created | 5 Nov 2012, 13:44:11 UTC |
| Sent | 5 Nov 2012, 13:44:19 UTC |
| Report deadline | 4 Feb 2013, 21:11:30 UTC |
| Received | 27 Dec 2012, 12:27:37 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1143633 |
| Run time | 16 days 11 hours 10 min 5 sec |
| CPU time | 16 days 11 hours 10 min 5 sec |
| Validate state | Invalid |
| Credit | 6,220.80 |
| Device peak FLOPS | 1.93 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>6.2.14</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/u41kko.pji1c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kko.pji1c10 Error: Input file: dataout/u41kko.pii1c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kko.pii1c10 Error: Input file: dataout/u41kko.pfi1c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kko.pfi1c10 Error: Input file: dataout/u41kka.phi1c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kka.phi1c10 Error: Input file: dataout/u41kka.pgi1c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kka.pgi1c10 Error: Input file: dataout/u41kka.pei1c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kka.pei1c10 Error: Input file: dataout/u41kka.pdi1c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kka.pdi1c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15516, selfPID=15516, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/u41kko.pji5c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kko.pji5c10 Error: Input file: dataout/u41kko.pii5c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kko.pii5c10 Error: Input file: dataout/u41kko.pfi5c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kko.pfi5c10 Error: Input file: dataout/u41kka.phi5c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kka.phi5c10 Error: Input file: dataout/u41kka.pgi5c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kka.pgi5c10 Error: Input file: dataout/u41kka.pei5c10 is not a valid UM file. Error converting file to netcdf: dataout/u41kka.pei5c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... *** glibc detected *** hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x087f2ff0 *** ======= Backtrace: ========= /lib/i686/cmov/libc.so.6[0xb756a764] /lib/i686/cmov/libc.so.6(cfree+0x96)[0xb756c966] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb77422e1] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xb774233d] hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb7512455] hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 09:00 1998871 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 09:00 1998871 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 080e4000 00:00 0 0879d000-08803000 rw-p 0879d000 00:00 0 [heap] b6f00000-b6f21000 rw-p b6f00000 00:00 0 b6f21000-b7000000 ---p b6f21000 00:00 0 b707b000-b74f6000 rw-s 00000000 09:00 3326712 /var/lib/boinc-client/slots/3/134735 b74f6000-b74f7000 ---p b74f6000 00:00 0 b74f7000-b74fc000 rw-p b74f7000 00:00 0 b74fc000-b7651000 r-xp 00000000 09:00 893027 /lib/i686/cmov/libc-2.7.so b7651000-b7652000 r--p 00155000 09:00 893027 /lib/i686/cmov/libc-2.7.so b7652000-b7654000 rw-p 00156000 09:00 893027 /lib/i686/cmov/libc-2.7.so b7654000-b7657000 rw-p b7654000 00:00 0 b7657000-b7663000 r-xp 00000000 09:00 884774 /lib/libgcc_s.so.1 b7663000-b7664000 rw-p 0000b000 09:00 884774 /lib/libgcc_s.so.1 b7664000-b7688000 r-xp 00000000 09:00 892996 /lib/i686/cmov/libm-2.7.so b7688000-b768a000 rw-p 00023000 09:00 892996 /lib/i686/cmov/libm-2.7.so b768a000-b776d000 r-xp 00000000 09:00 3948705 /usr/lib/libstdc++.so.6.0.10 b776d000-b7770000 r--p 000e2000 09:00 3948705 /usr/lib/libstdc++.so.6.0.10 b7770000-b7772000 rw-p 000e5000 09:00 3948705 /usr/lib/libstdc++.so.6.0.10 b7772000-b7779000 rw-p b7772000 00:00 0 b7779000-b777b000 r-xp 00000000 09:00 893018 /lib/i686/cmov/libdl-2.7.so b777b000-b777d000 rw-p 00001000 09:00 893018 /lib/i686/cmov/libdl-2.7.so b777d000-b7792000 r-xp 00000000 09:00 893002 /lib/i686/cmov/libpthread-2.7.so b7792000-b7794000 rw-p 00014000 09:00 893002 /lib/i686/cmov/libpthread-2.7.so b7794000-b7796000 rw-p b7794000 00:00 0 b7797000-b7798000 rw-p b7797000 00:00 0 b7798000-b779a000 rw-s 00000000 09:00 3326707 /var/lib/boinc-client/slots/3/boinc_mmap_file b779a000-b779c000 rw-p b779a000 00:00 0 b779c000-b779d000 r-xp b779c000 00:00 0 [vdso] b779d000-b77b7000 r-xp 00000000 09:00 885607 /lib/ld-2.7.so b77b7000-b77b9000 rw-p 0001a000 09:00 885607 /lib/ld-2.7.so bf991000-bfa01000 rw-p bff8f000 00:00 0 [stack] SIGABRT: abort called Stack trace (18 frames): hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xb779c400] [0xb779c424] /lib/i686/cmov/libc.so.6(gsignal+0x50)[0xb7527640] /lib/i686/cmov/libc.so.6(abort+0x188)[0xb7529018] /lib/i686/cmov/libc.so.6[0xb756448d] /lib/i686/cmov/libc.so.6[0xb756a764] /lib/i686/cmov/libc.so.6(cfree+0x96)[0xb756c966] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb77422e1] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xb774233d] hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb7512455] hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 26 Dec 2012 12:29:43 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 518,400 | 1,422,600 | 2.7442 |
| 24 Dec 2012 11:28:31 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 492,480 | 1,352,138 | 2.7456 |
| 23 Dec 2012 02:31:44 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 466,560 | 1,281,251 | 2.7462 |
| 21 Dec 2012 15:22:03 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 440,640 | 1,210,062 | 2.7461 |
| 19 Dec 2012 21:53:46 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 414,720 | 1,137,565 | 2.7430 |
| 18 Dec 2012 09:01:41 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 388,800 | 1,066,148 | 2.7422 |
| 16 Dec 2012 23:27:53 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 362,880 | 995,530 | 2.7434 |
| 15 Dec 2012 18:21:35 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 336,960 | 925,221 | 2.7458 |
| 14 Dec 2012 16:15:56 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 311,040 | 854,015 | 2.7457 |
| 14 Dec 2012 16:15:56 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 285,120 | 783,890 | 2.7493 |
| 14 Dec 2012 16:15:56 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 259,200 | 712,961 | 2.7506 |
| 14 Dec 2012 16:15:56 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 233,280 | 642,354 | 2.7536 |
| 07 Dec 2012 15:43:12 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 207,360 | 572,712 | 2.7619 |
| 05 Dec 2012 23:46:36 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 181,440 | 502,301 | 2.7684 |
| 04 Dec 2012 19:04:01 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 155,520 | 432,293 | 2.7797 |
| 03 Dec 2012 14:20:37 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 129,600 | 362,002 | 2.7932 |
| 02 Dec 2012 09:23:44 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 103,680 | 292,609 | 2.8222 |
| 26 Nov 2012 16:02:30 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 77,760 | 220,797 | 2.8395 |
| 19 Nov 2012 21:24:19 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 51,840 | 147,070 | 2.8370 |
| 13 Nov 2012 02:37:34 | 1143633 | 15427059 | hadcm3n_u41k_1980_40_008244098_1 | 25,920 | 73,533 | 2.8369 |
©2024 climateprediction.net