Task 13114929

Name	hadcm3n_yhhr_1900_40_007355529_1
Workunit	7552959
Created	6 Jul 2011, 14:40:41 UTC
Sent	10 Jul 2011, 8:06:16 UTC
Report deadline	9 Oct 2011, 15:33:27 UTC
Received	12 Sep 2011, 20:00:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1073659
Run time	26 days 7 hours 31 min 11 sec
CPU time	25 days 6 hours 49 min 14 sec
Validate state	Invalid
Credit	7,153.92
Device peak FLOPS	1.23 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf7790400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf74ebc76] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf779e400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf74f9c76] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... SIGABRT: abort called Stack trace (10 frames): /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf77aa400] [0xf77aa430] /lib32/libc.so.6(gsignal+0x51)[0xf762ca21] /lib32/libc.so.6(abort+0x182)[0xf762fe42] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7618c76] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3854, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7725400] [0xf7725430] /lib32/libc.so.6(gsignal+0x51)[0xf75a7a21] /lib32/libc.so.6(abort+0x182)[0xf75aae42] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7593c76] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3854, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf77c7400] [0xf77c7430] /lib32/libc.so.6(gsignal+0x51)[0xf7649a21] /lib32/libc.so.6(abort+0x182)[0xf764ce42] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7635c76] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3854, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf771a400] [0xf771a430] /lib32/libc.so.6(gsignal+0x51)[0xf759ca21] /lib32/libc.so.6(abort+0x182)[0xf759fe42] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7588c76] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3854, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf773a400] [0xf773a430] /lib32/libc.so.6(gsignal+0x51)[0xf75bca21] /lib32/libc.so.6(abort+0x182)[0xf75bfe42] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75a8c76] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3854, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (10 frames): /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7728400] [0xf7728430] /lib32/libc.so.6(gsignal+0x51)[0xf75aaa21] /lib32/libc.so.6(abort+0x182)[0xf75ade42] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7596c76] /home/monique/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3854, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Sep 2011 14:02:17	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	596,160	2,205,648	3.6998
09 Sep 2011 22:35:28	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	570,240	2,109,773	3.6998
08 Sep 2011 11:18:46	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	544,320	2,014,126	3.7003
04 Sep 2011 02:21:07	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	518,400	1,918,902	3.7016
02 Sep 2011 22:14:26	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	492,480	1,823,717	3.7031
01 Sep 2011 18:36:14	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	466,560	1,728,764	3.7053
30 Aug 2011 09:44:01	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	440,640	1,633,411	3.7069
23 Aug 2011 22:12:28	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	414,720	1,537,781	3.7080
19 Aug 2011 16:23:59	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	388,800	1,442,201	3.7094
15 Aug 2011 19:17:09	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	362,880	1,347,744	3.7140
14 Aug 2011 06:56:02	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	336,960	1,253,408	3.7198
30 Jul 2011 14:06:40	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	311,040	1,157,649	3.7219
30 Jul 2011 14:06:40	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	285,120	1,062,201	3.7255
30 Jul 2011 14:06:40	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	259,200	966,599	3.7292
30 Jul 2011 14:06:40	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	233,280	870,801	3.7329
30 Jul 2011 14:06:40	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	207,360	775,450	3.7396
30 Jul 2011 14:06:40	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	181,440	680,088	3.7483
30 Jul 2011 14:06:40	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	155,520	585,087	3.7621
25 Jul 2011 15:52:22	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	129,600	488,306	3.7678
25 Jul 2011 14:42:06	1073659	13114929	hadcm3n_yhhr_1900_40_007355529_1	103,680	390,575	3.7671