Task 17520781

Name	hadam3pm2_k0a6_1959_10_009209663_2
Workunit	9335285
Created	30 Nov 2014, 17:54:20 UTC
Sent	30 Nov 2014, 18:09:53 UTC
Report deadline	12 Nov 2015, 23:29:53 UTC
Received	25 Dec 2014, 8:43:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1343229
Run time	6 days 3 hours 56 min 40 sec
CPU time	4 days 3 hours 33 min 32 sec
Validate state	Invalid
Credit	3,577.84
Device peak FLOPS	0.80 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> SIGSEGV: segmentation violation Stack trace (13 frames): /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x482345b5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26143, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... 02:44:51 (28239): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (13 frames): /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x482345b5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4769, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (13 frames): /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x482345b5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... Signal 1 received, exiting... 14:47:21 (7793): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... Signal 1 received, exiting... 02:54:06 (9407): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... 02:36:42 (10030): called boinc_finish Signal 1 received, exiting... 02:36:42 (10021): called boinc_finish CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (13 frames): /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x482345b5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8687, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Stack trace (13 frames): /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /media/sda5/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x482345b5] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8687, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... 02:55:58 (28574): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 1 received, exiting... Signal 1 received, exiting... 02:47:55 (10140): called boinc_finish CPDN Monitor - Quit request from BOINC... Model crashed: § Model crashed: § Model crashed: § Model crashed: § Model crashed: § Model crashed: § Sorry, too many model crashes! :-( 10:58:51 (15805): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Dec 2014 06:12:35	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	123,908	356,389	2.8762
25 Dec 2014 03:31:58	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	121,028	347,443	2.8708
25 Dec 2014 00:54:24	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	118,148	338,534	2.8653
24 Dec 2014 22:49:04	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	115,268	329,511	2.8587
24 Dec 2014 19:33:36	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	112,388	320,580	2.8524
24 Dec 2014 16:58:07	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	109,508	311,623	2.8457
24 Dec 2014 14:27:41	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	106,628	302,628	2.8382
21 Dec 2014 22:38:42	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	89,348	226,149	2.5311
21 Dec 2014 19:18:09	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	86,468	214,658	2.4825
21 Dec 2014 16:07:34	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	83,588	203,828	2.4385
21 Dec 2014 12:52:03	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	80,708	312,596	3.8732
21 Dec 2014 09:41:29	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	77,828	301,810	3.8779
21 Dec 2014 06:25:52	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	74,948	290,996	3.8826
21 Dec 2014 03:20:02	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	72,068	280,564	3.8930
18 Dec 2014 22:11:54	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	40,388	167,039	4.1359
18 Dec 2014 18:26:17	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	37,508	154,325	4.1145
30 Nov 2014 22:09:30	1343229	17520781	hadam3pm2_k0a6_1959_10_009209663_2	2,948	8,549	2.8999