Task 19420540

Name	hadcm3n_sd0r_198012_480_375_010414870_0
Workunit	10414870
Created	21 Mar 2016, 16:48:17 UTC
Sent	22 Mar 2016, 14:51:08 UTC
Report deadline	4 Mar 2017, 20:11:08 UTC
Received	14 May 2016, 23:08:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1354552
Run time	8 days 2 hours 48 min 36 sec
CPU time	4 days 22 hours 46 min 26 sec
Validate state	Invalid
Credit	5,909.76
Device peak FLOPS	3.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Stack trace (13 frames): /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77c0e50] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf75c5993] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15863, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7747e50] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf754c993] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15863, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77c1e50] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf75c6993] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15863, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7723e50] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf7528993] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15863, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77a4e50] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf75a9993] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15863, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf76e7e50] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xf3)[0xf74ec993] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15863, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 May 2016 21:06:45	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	492,480	665,409	1.3511
14 May 2016 09:49:07	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	466,560	627,317	1.3446
13 May 2016 23:10:28	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	440,640	590,914	1.3410
13 May 2016 12:12:35	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	414,720	554,545	1.3372
13 May 2016 04:52:06	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	388,800	523,180	1.3456
13 May 2016 04:52:06	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	362,880	499,903	1.3776
04 May 2016 06:08:09	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	336,960	471,656	1.3997
27 Apr 2016 02:52:21	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	311,040	436,374	1.4030
22 Apr 2016 07:06:04	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	285,120	410,615	1.4401
22 Apr 2016 07:06:04	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	259,200	387,472	1.4949
22 Apr 2016 07:06:04	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	233,280	364,744	1.5635
15 Apr 2016 12:17:00	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	207,360	331,326	1.5978
15 Apr 2016 12:17:00	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	181,440	290,236	1.5996
15 Apr 2016 12:06:12	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	155,520	249,133	1.6019
08 Apr 2016 17:46:07	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	129,600	206,273	1.5916
30 Mar 2016 14:59:58	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	103,680	165,480	1.5961
30 Mar 2016 03:02:08	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	77,760	123,955	1.5941
29 Mar 2016 15:06:52	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	51,840	82,348	1.5885
28 Mar 2016 18:36:04	1354552	19420540	hadcm3n_sd0r_198012_480_375_010414870_0	25,920	40,906	1.5782