Name | hadam4h_h0vy_201311_5_889_012044637_2 |
Workunit | 12044637 |
Created | 19 Feb 2021, 0:55:21 UTC |
Sent | 19 Feb 2021, 1:38:57 UTC |
Report deadline | 1 Feb 2022, 6:58:57 UTC |
Received | 22 Mar 2021, 14:54:40 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 1493612 |
Run time | 14 days 18 hours 53 min 56 sec |
CPU time | 14 days 9 hours 31 min 23 sec |
Validate state | Invalid |
Credit | 27,115.14 |
Device peak FLOPS | 7.17 GFLOPS |
Application version | UK Met Office HadAM4 at N216 resolution v8.52 i686-pc-linux-gnu |
Peak working set size | 1,377.30 MB |
Peak swap size | 1,398.28 MB |
Peak disk usage | 163.14 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63)</message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:30:06 (46482): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f0e580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a07552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7a8bd2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7a8c0ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7a8a03b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f0e580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f0e569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7a96693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7a96892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7a9d314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7ad1475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf79eeee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 20:39:19 (7242): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f8a580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a83552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b07d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b080ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b0603b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f8a580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f8a569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b12693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b12892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b19314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b4d475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a6aee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fa2580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a9b552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b1fd2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b200ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b1e03b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fa2580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fa2569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b2a693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b2a892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b31314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b65475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a82ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f86580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a7f552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b03d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b040ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b0203b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f86580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f86569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b0e693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b0e892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b15314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b49475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a66ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f56580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a4f552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7ad3d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7ad40ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7ad203b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f56580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f56569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7ade693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7ade892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7ae5314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b19475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a36ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f52580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a4b552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7acfd2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7ad00ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7ace03b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f52580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f52569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7ada693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7ada892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7ae1314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b15475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a32ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 07:37:57 (10283): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:49:09 (3701): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7ef4580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf79ed552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7a71d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7a720ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7a7003b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7ef4580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7ef4569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7a7c693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7a7c892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7a83314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7ab7475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf79d4ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f7c580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a75552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7af9d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7afa0ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7af803b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f7c580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f7c569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b04693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b04892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b0b314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b3f475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a5cee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 11:54:11 (7280): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f73580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a6c552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7af0d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7af10ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7aef03b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f73580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f73569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7afb693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7afb892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b02314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b36475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a53ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:28:45 (9136): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb2580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7aab552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b2fd2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b300ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b2e03b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb2580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fb2569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b3a693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b3a892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b41314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b75475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a92ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:21:02 (4144): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 13:21:03 (4144): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f3f580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a38552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7abcd2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7abd0ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7abb03b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f3f580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f3f569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7ac7693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7ac7892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7ace314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b02475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a1fee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb4580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7aad552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b31d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b320ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b3003b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb4580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fb4569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b3c693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b3c892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b43314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b77475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a94ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:11:38 (5476): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 08:11:38 (5476): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f97580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a90552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b14d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b150ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b1303b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f97580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f97569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b1f693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b1f892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b26314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b5a475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a77ee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f8c580] /lib/i386-linux-gnu/libc.so.6(getenv+0xa2)[0xf7a85552] /lib/i386-linux-gnu/libc.so.6(+0xbbd2d)[0xf7b09d2d] /lib/i386-linux-gnu/libc.so.6(+0xbc0ee)[0xf7b0a0ee] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1b)[0xf7b0803b] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f8c580] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f8c569] /lib/i386-linux-gnu/libc.so.6(+0xc6693)[0xf7b14693] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x52)[0xf7b14892] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x24)[0xf7b1b314] /lib/i386-linux-gnu/libc.so.6(usleep+0x45)[0xf7b4f475] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xf7a6cee5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
19 Mar 2021 00:30:52 | 1493612 | 22019328 | hadam4h_h0vy_201311_5_889_012044637_2 | 34,763 | 1,097,558 | 31.5726 |
15 Mar 2021 17:05:17 | 1493612 | 22019328 | hadam4h_h0vy_201311_5_889_012044637_2 | 26,123 | 900,433 | 34.4690 |
11 Mar 2021 03:03:25 | 1493612 | 22019328 | hadam4h_h0vy_201311_5_889_012044637_2 | 17,483 | 666,962 | 38.1492 |
04 Mar 2021 05:10:37 | 1493612 | 22019328 | hadam4h_h0vy_201311_5_889_012044637_2 | 8,843 | 408,844 | 46.2336 |
©2024 climateprediction.net