Task 17954971

Name	hadam3pm2_pfaj_1991_10_009530116_0
Workunit	9611859
Created	13 Feb 2015, 16:55:51 UTC
Sent	9 Mar 2015, 21:23:37 UTC
Report deadline	20 Feb 2016, 2:43:37 UTC
Received	26 Mar 2015, 21:38:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1337590
Run time	11 days 15 hours 38 min 45 sec
CPU time	9 days 10 hours 45 min 56 sec
Validate state	Invalid
Credit	7,236.88
Device peak FLOPS	1.59 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 01:12:06 (7552): called boinc_finish Signal 15 received, exiting... 01:12:06 (7552): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 00:46:46 (3193): called boinc_finish Signal 15 received, exiting... 00:46:46 (3193): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 23:39:38 (4313): called boinc_finish Signal 15 received, exiting... 23:39:38 (4313): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 00:44:56 (3193): called boinc_finish Signal 15 received, exiting... 00:44:56 (3193): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 01:40:37 (4639): called boinc_finish Signal 15 received, exiting... 01:40:37 (4639): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 01:11:45 (7117): called boinc_finish Signal 15 received, exiting... 01:11:45 (7117): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Stack trace (13 frames): /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] [0xf0f25bc0] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x813f937] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8136094] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8129ec4] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8129cab] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x81320ba] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8077a4f] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x808dc3c] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /usr/lib/libc.so.6(__libc_start_main+0xde)[0x42c57e7e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4035, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 21:51:38 (10561): called boinc_finish Signal 15 received, exiting... 21:51:38 (10561): called boinc_finish oa.pc\|2dec.nc oa.pc\|3jan.nc oa.pc\|3feb.nc oa.pc\|3mar.nc oa.pc\|3apr.nc oa.pc\|3may.nc oa.pc\|3jun.nc oa.pc\|3jul.nc oa.pc\|3aug.nc oa.pc\|3sep.nc oa.pc\|3oct.nc oa.pc\|3nov.nc oa.pe\|2dec.nc oa.pe\|3jan.nc oa.pe\|3feb.nc oa.pe\|3mar.nc oa.pe\|3apr.nc oa.pe\|3may.nc oa.pe\|3jun.nc oa.pe\|3jul.nc oa.pe\|3aug.nc oa.pe\|3sep.nc oa.pe\|3oct.nc oa.pe\|3nov.nc SIGSEGV: segmentation violation Stack trace (15 frames): /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] [0xf0f21bc0] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x82a4308] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x830ca0c] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8201c5a] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x827a827] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x81488a8] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x81c1a69] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8250b34] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8081970] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /usr/lib/libc.so.6(__libc_start_main+0xde)[0x42c57e7e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3003, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|3dec.nc oa.pc\|4jan.nc oa.pc\|4feb.nc oa.pc\|4mar.nc oa.pc\|4apr.nc oa.pc\|4may.nc oa.pc\|4jun.nc oa.pc\|4jul.nc oa.pc\|4aug.nc oa.pc\|4sep.nc oa.pc\|4oct.nc oa.pc\|4nov.nc oa.pe\|3dec.nc oa.pe\|4jan.nc oa.pe\|4feb.nc oa.pe\|4mar.nc oa.pe\|4apr.nc oa.pe\|4may.nc oa.pe\|4jun.nc oa.pe\|4jul.nc oa.pe\|4aug.nc oa.pe\|4sep.nc oa.pe\|4oct.nc oa.pe\|4nov.nc Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|4dec.nc oa.pc\|5jan.nc oa.pc\|5feb.nc oa.pc\|5mar.nc oa.pc\|5apr.nc oa.pc\|5may.nc oa.pc\|5jun.nc oa.pc\|5jul.nc oa.pc\|5aug.nc oa.pc\|5sep.nc oa.pc\|5oct.nc oa.pc\|5nov.nc oa.pe\|4dec.nc oa.pe\|5jan.nc oa.pe\|5feb.nc oa.pe\|5mar.nc oa.pe\|5apr.nc oa.pe\|5may.nc oa.pe\|5jun.nc oa.pe\|5jul.nc oa.pe\|5aug.nc oa.pe\|5sep.nc oa.pe\|5oct.nc oa.pe\|5nov.nc Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 23:12:39 (28758): called boinc_finish Signal 15 received, exiting... 23:12:40 (28758): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 17:55:05 (3349): called boinc_finish Signal 15 received, exiting... 17:55:05 (3349): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 23:48:14 (2890): called boinc_finish Signal 15 received, exiting... 23:48:14 (2890): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Stack trace (11 frames): /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] [0xf0f13bc0] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8147b3f] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x81c1a69] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8250b34] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8081970] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/data/var/lib/boinc/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /usr/lib/libc.so.6(__libc_start_main+0xde)[0x42c57e7e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2947, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 00:51:12 (3000): called boinc_finish Signal 15 received, exiting... 00:51:12 (3000): called boinc_finish oa.pc\|7dec.nc oa.pc\|8jan.nc oa.pc\|8feb.nc oa.pc\|8mar.nc oa.pc\|8apr.nc oa.pc\|8may.nc oa.pc\|8jun.nc oa.pc\|8jul.nc oa.pc\|8aug.nc oa.pc\|8sep.nc oa.pc\|8oct.nc oa.pc\|8nov.nc oa.pe\|7dec.nc oa.pe\|8jan.nc oa.pe\|8feb.nc oa.pe\|8mar.nc oa.pe\|8apr.nc oa.pe\|8may.nc oa.pe\|8jun.nc oa.pe\|8jul.nc oa.pe\|8aug.nc oa.pe\|8sep.nc oa.pe\|8oct.nc oa.pe\|8nov.nc Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 22:41:48 (3425): called boinc_finish Signal 15 received, exiting... 22:41:48 (3425): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... 21:52:01 (3163): called boinc_finish Signal 15 received, exiting... 21:52:01 (3163): called boinc_finish Signal 3 received, exiting... 22:36:34 (3157): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Mar 2015 09:27:46	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	250,628	727,133	2.9012
26 Mar 2015 09:27:11	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	247,748	716,700	2.8929
26 Mar 2015 09:26:14	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	244,868	705,880	2.8827
26 Mar 2015 09:23:16	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	239,108	684,111	2.8611
26 Mar 2015 09:23:16	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	236,228	673,037	2.8491
26 Mar 2015 09:23:16	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	233,348	662,086	2.8373
26 Mar 2015 09:23:16	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	230,468	651,002	2.8247
26 Mar 2015 09:23:16	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	227,588	640,011	2.8121
26 Mar 2015 09:23:16	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	224,708	629,055	2.7994
26 Mar 2015 09:23:16	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	221,828	618,167	2.7867
23 Mar 2015 13:09:57	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	218,948	607,201	2.7733
23 Mar 2015 10:08:57	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	216,068	596,308	2.7598
23 Mar 2015 07:18:08	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	213,188	585,547	2.7466
23 Mar 2015 04:15:57	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	210,308	574,784	2.7331
21 Mar 2015 15:12:30	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	184,388	510,069	2.7663
21 Mar 2015 12:16:51	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	181,508	499,685	2.7530
21 Mar 2015 09:21:12	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	178,628	489,292	2.7392
21 Mar 2015 06:25:26	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	175,748	478,932	2.7251
20 Mar 2015 21:37:47	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	169,988	458,768	2.6988
20 Mar 2015 18:31:47	1337590	17954971	hadam3pm2_pfaj_1991_10_009530116_0	167,108	448,419	2.6834