Task 22160156

Name	hadam4h_20qo_209502_4_922_012120378_2
Workunit	12120378
Created	12 Nov 2021, 3:40:23 UTC
Sent	12 Nov 2021, 21:09:17 UTC
Report deadline	26 Oct 2022, 2:29:17 UTC
Received	10 Dec 2021, 14:41:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	3 (0x00000003) Unknown error code
Computer ID	1524007
Run time	22 days 7 hours 0 min 9 sec
CPU time	22 days 4 hours 53 min 56 sec
Validate state	Invalid
Credit	13,636.74
Device peak FLOPS	1.00 GFLOPS
Application version	UK Met Office HadAM4 at N216 resolution v8.52 i686-pc-linux-gnu
Peak working set size	1,377.49 MB
Peak swap size	1,398.62 MB
Peak disk usage	12.93 MB
Stderr	<core_client_version>7.16.17</core_client_version> <![CDATA[ <message> process exited with code 3 (0x3, -253)</message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 23:22:51 (1073871): called boinc_finish(193) BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 16:37:10 (1242253): called boinc_finish(193) Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb8560] /lib/libc.so.6(getenv+0x9a)[0xf7ac89fa] /lib/libc.so.6(+0xb705d)[0xf7b4c05d] /lib/libc.so.6(+0xb740a)[0xf7b4c40a] /lib/libc.so.6(localtime_r+0x17)[0xf7b49ea7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb8560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fb8549] /lib/libc.so.6(+0xc1902)[0xf7b56902] /lib/libc.so.6(nanosleep+0x5e)[0xf7b5caee] /lib/libc.so.6(usleep+0x41)[0xf7b904c1] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/libc.so.6(__libc_start_main+0xe5)[0xf7ab09a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: STWORK : Error in PP_FILE tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 04:21:42 (1292913): called boinc_finish(193) BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/20qoga.dat53r6 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/xnnuj.pipe_dummy Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f6d560] /lib/libc.so.6(getenv+0x9a)[0xf7a7d9fa] /lib/libc.so.6(+0xb705d)[0xf7b0105d] /lib/libc.so.6(+0xb740a)[0xf7b0140a] /lib/libc.so.6(localtime_r+0x17)[0xf7afeea7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f6d560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f6d549] /lib/libc.so.6(+0xc1902)[0xf7b0b902] /lib/libc.so.6(nanosleep+0x5e)[0xf7b11aee] /lib/libc.so.6(usleep+0x41)[0xf7b454c1] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/libc.so.6(__libc_start_main+0xe5)[0xf7a659a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: STWORK : Error in PP_FILE tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: STWORK : Error in PP_FILE tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f43560] /lib/libc.so.6(getenv+0x9a)[0xf7a539fa] /lib/libc.so.6(+0xb705d)[0xf7ad705d] /lib/libc.so.6(+0xb740a)[0xf7ad740a] /lib/libc.so.6(localtime_r+0x17)[0xf7ad4ea7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f43560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f43549] /lib/libc.so.6(+0xc1902)[0xf7ae1902] /lib/libc.so.6(nanosleep+0x5e)[0xf7ae7aee] /lib/libc.so.6(usleep+0x41)[0xf7b1b4c1] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/libc.so.6(__libc_start_main+0xe5)[0xf7a3b9a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 17:14:38 (1352007): called boinc_finish(193) BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f66560] /lib/libc.so.6(getenv+0x9a)[0xf7a769fa] /lib/libc.so.6(+0xb705d)[0xf7afa05d] /lib/libc.so.6(+0xb740a)[0xf7afa40a] /lib/libc.so.6(localtime_r+0x17)[0xf7af7ea7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f66560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f66549] /lib/libc.so.6(+0xc1902)[0xf7b04902] /lib/libc.so.6(nanosleep+0x5e)[0xf7b0aaee] /lib/libc.so.6(usleep+0x41)[0xf7b3e4c1] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/libc.so.6(__libc_start_main+0xe5)[0xf7a5e9a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f18560] /lib/libc.so.6(getenv+0x9a)[0xf7a289fa] /lib/libc.so.6(+0xb705d)[0xf7aac05d] /lib/libc.so.6(+0xb740a)[0xf7aac40a] /lib/libc.so.6(localtime_r+0x17)[0xf7aa9ea7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f18560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f18549] /lib/libc.so.6(+0xc1902)[0xf7ab6902] /lib/libc.so.6(nanosleep+0x5e)[0xf7abcaee] /lib/libc.so.6(usleep+0x41)[0xf7af04c1] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/libc.so.6(__libc_start_main+0xe5)[0xf7a109a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fbd560] /lib/libc.so.6(getenv+0x9a)[0xf7acd9fa] /lib/libc.so.6(+0xb705d)[0xf7b5105d] /lib/libc.so.6(+0xb740a)[0xf7b5140a] /lib/libc.so.6(localtime_r+0x17)[0xf7b4eea7] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fbd560] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fbd549] /lib/libc.so.6(+0xc1902)[0xf7b5b902] /lib/libc.so.6(nanosleep+0x5e)[0xf7b61aee] /lib/libc.so.6(usleep+0x41)[0xf7b954c1] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b] /lib/libc.so.6(__libc_start_main+0xe5)[0xf7ab59a5] ../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21] Exiting... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Dec 2021 12:37:28	1524007	22160156	hadam4h_20qo_209502_4_922_012120378_2	17,483	1,324,020	75.7319
25 Nov 2021 06:29:18	1524007	22160156	hadam4h_20qo_209502_4_922_012120378_2	8,843	620,358	70.1524