Task 19957822

Name	wah2_sas50_kut8_199812_13_448_010713648_0
Workunit	10713648
Created	12 Sep 2016, 14:48:10 UTC
Sent	19 Sep 2016, 14:39:22 UTC
Report deadline	1 Sep 2017, 19:59:22 UTC
Received	25 Sep 2016, 1:34:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1377172
Run time	1 days 7 hours 2 min 57 sec
CPU time	1 days 4 hours 19 min 16 sec
Validate state	Invalid
Credit	779.66
Device peak FLOPS	2.42 GFLOPS
Application version	Weather At Home 2 (wah2) (region independent) v8.12 i686-pc-linux-gnu
Peak working set size	237.00 MB
Peak swap size	272.53 MB
Peak disk usage	92.00 MB
Stderr	<core_client_version>7.6.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 13:13:43 (19746): called boinc_finish(193) Signal 15 received, exiting... 13:13:43 (19747): called boinc_finish(193) Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (20 frames): ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c9f4f] [0xb778b404] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xb73b116a] /lib/i386-linux-gnu/libc.so.6(+0xa229b)[0xb742529b] /lib/i386-linux-gnu/libc.so.6(+0xa260a)[0xb742560a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1e)[0xb7423c9e] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c4d96] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c528c] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c5391] [0xb778b404] [0xb778b428] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4a)[0xb743333a] /lib/i386-linux-gnu/libc.so.6(usleep+0x35)[0xb7463385] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80dda50] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x8050953] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x8051103] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80523d4] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x8052674] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xb739b637] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d4d1] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... 20:08:44 (1423): called boinc_finish(193) 20:08:44 (1422): called boinc_finish(193) Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (19 frames): ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c9f4f] [0xb776d404] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xb739316a] /lib/i386-linux-gnu/libc.so.6(+0xa229b)[0xb740729b] /lib/i386-linux-gnu/libc.so.6(+0xa260a)[0xb740760a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1e)[0xb7405c9e] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c4d96] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c528c] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c5391] [0xb776d404] [0xb776d428] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4a)[0xb741533a] /lib/i386-linux-gnu/libc.so.6(usleep+0x35)[0xb7445385] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80dda50] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x8051114] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80523d4] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x8052674] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xb737d637] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d4d1] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 forrtl: No space left on device forrtl: severe (38): error during write, unit 0, file /var/lib/boinc-client/projects/climateprediction.net/wah2_sas50_kut8_199812_13_448_010713648/dataout/xacxf.err Image PC Routine Line Source wah2rm3m2t_um_8.1 083F37DD Unknown Unknown Unknown wah2rm3m2t_um_8.1 083F25D5 Unknown Unknown Unknown wah2rm3m2t_um_8.1 083C302F Unknown Unknown Unknown wah2rm3m2t_um_8.1 0837F19D Unknown Unknown Unknown wah2rm3m2t_um_8.1 0837EAE7 Unknown Unknown Unknown wah2rm3m2t_um_8.1 083B911F Unknown Unknown Unknown wah2rm3m2t_um_8.1 083B5B59 Unknown Unknown Unknown wah2rm3m2t_um_8.1 082EA0E8 Unknown Unknown Unknown wah2rm3m2t_um_8.1 082ECD6C Unknown Unknown Unknown libc.so.6 4017C637 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1263, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... SIGSEGV: segmentation violation Stack trace (14 frames): ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80c9f4f] [0xb7768404] /lib/i386-linux-gnu/libc.so.6(getenv+0x9a)[0xb738e16a] /lib/i386-linux-gnu/libc.so.6(+0xa229b)[0xb740229b] /lib/i386-linux-gnu/libc.so.6(+0xa260a)[0xb740260a] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x1e)[0xb7400c9e] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c4d96] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c528c] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x80c5391] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x8050463] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x805242c] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu[0x8052674] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xb7378637] ../../projects/climateprediction.net/wah2_8.12_i686-pc-linux-gnu(__gxx_personality_v0+0x1bd)[0x804d4d1] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Sep 2016 13:40:57	1377172	19957822	wah2_sas50_kut8_199812_13_448_010713648_0	11,819	53,010	4.4852