Task 22142654

Name	hadam4h_a0t5_209411_5_891_012050136_3
Workunit	12050136
Created	3 Sep 2021, 13:30:55 UTC
Sent	4 Sep 2021, 5:20:45 UTC
Report deadline	17 Aug 2022, 10:40:45 UTC
Received	6 Oct 2021, 22:16:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1494452
Run time	13 days 7 hours 40 min 8 sec
CPU time	13 days 3 hours 48 min 52 sec
Validate state	Invalid
Credit	20,375.94
Device peak FLOPS	7.12 GFLOPS
Application version	UK Met Office HadAM4 at N216 resolution v8.52 i686-pc-linux-gnu
Peak working set size	1,366.66 MB
Peak swap size	1,382.67 MB
Peak disk usage	162.60 MB
Stderr	<core_client_version>7.16.17</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63)</message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:24:44 (3487): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:24:45 (3487): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f01570] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf79d3e0a] /lib/i386-linux-gnu/libc.so.6(+0xb9f7d)[0xf7a56f7d] /lib/i386-linux-gnu/libc.so.6(+0xba32a)[0xf7a5732a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7a55277] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f01570] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f01559] /lib/i386-linux-gnu/libc.so.6(+0xc43ed)[0xf7a613ed] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x4e)[0xf7a614be] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x20)[0xf7a67a20] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7a9a051] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0x106)[0xf79bbe46] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:58:02 (13948): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:58:03 (13948): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 20:58:51 (4212): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f30570] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf7a08e0a] /lib/i386-linux-gnu/libc.so.6(+0xb9f7d)[0xf7a8bf7d] /lib/i386-linux-gnu/libc.so.6(+0xba32a)[0xf7a8c32a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7a8a277] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f30570] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f30559] /lib/i386-linux-gnu/libc.so.6(+0xc43ed)[0xf7a963ed] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x4e)[0xf7a964be] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x20)[0xf7a9ca20] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7acf051] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0x106)[0xf79f0e46] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f6e570] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf7a46e0a] /lib/i386-linux-gnu/libc.so.6(+0xb9f7d)[0xf7ac9f7d] /lib/i386-linux-gnu/libc.so.6(+0xba32a)[0xf7aca32a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7ac8277] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f6e570] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f6e559] /lib/i386-linux-gnu/libc.so.6(+0xc43ed)[0xf7ad43ed] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x4e)[0xf7ad44be] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x20)[0xf7adaa20] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7b0d051] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0x106)[0xf7a2ee46] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 20:22:02 (1531): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 20:22:03 (1531): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (22 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f65570] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf7a38f6a] /lib/i386-linux-gnu/libc.so.6(+0xb9add)[0xf7abbadd] /lib/i386-linux-gnu/libc.so.6(+0xb9e8a)[0xf7abbe8a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7ab9b17] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f65570] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f65559] /lib/i386-linux-gnu/libc.so.6(+0xc415d)[0xf7ac615d] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x4e)[0xf7ac622e] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x20)[0xf7acc770] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7b00181] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0x106)[0xf7a20fd6] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 01:26:41 (1680): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 01:26:42 (1680): called boinc_finish(193) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f03570] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf79d6f6a] /lib/i386-linux-gnu/libc.so.6(+0xb9add)[0xf7a59add] /lib/i386-linux-gnu/libc.so.6(+0xb9e8a)[0xf7a59e8a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7a57b17] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f03570] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f03559] /lib/i386-linux-gnu/libc.so.6(+0xc415d)[0xf7a6415d] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x4e)[0xf7a6422e] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x20)[0xf7a6a770] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7a9e181] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0x106)[0xf79befd6] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /more/var/lib/boinc-client/projects/climateprediction.net/hadam4h_a0t5_209411_5_891_012050136/dataout/xnnuj.out Image PC Routine Line Source hadam4_um_8.52_i6 083F6605 Unknown Unknown Unknown hadam4_um_8.Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 3 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7ed5570] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xf79a8f6a] /lib/i386-linux-gnu/libc.so.6(+0xb9add)[0xf7a2badd] /lib/i386-linux-gnu/libc.so.6(+0xb9e8a)[0xf7a2be8a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7a29b17] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7ed5570] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7ed5559] /lib/i386-linux-gnu/libc.so.6(+0xc415d)[0xf7a3615d] /lib/i386-linux-gnu/libc.so.6(clock_nanosleep+0x4e)[0xf7a3622e] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x20)[0xf7a3c770] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7a70181] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0x106)[0xf7990fd6] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Oct 2021 01:33:44	1494452	22142654	hadam4h_a0t5_209411_5_891_012050136_3	26,123	1,017,932	38.9669
28 Sep 2021 04:45:42	1494452	22142654	hadam4h_a0t5_209411_5_891_012050136_3	17,483	728,663	41.6784
15 Sep 2021 18:40:55	1494452	22142654	hadam4h_a0t5_209411_5_891_012050136_3	8,843	360,300	40.7441