Task 13028185

Name	hadcm3n_t7fe_1940_40_007317129_1
Workunit	7514559
Created	29 Jun 2011, 6:31:23 UTC
Sent	29 Jun 2011, 6:37:18 UTC
Report deadline	28 Sep 2011, 14:04:29 UTC
Received	20 Jul 2011, 13:06:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1153762
Run time	17 days 1 hours 38 min 41 sec
CPU time	15 days 13 hours 11 min 53 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	2.99 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.59</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (11 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf77cc400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804cf33] [0xf77cc400] [0xf77cc425] /lib32/libc.so.6(nanosleep+0x46)[0xf758f9c6] /lib32/libc.so.6(usleep+0x3c)[0xf75c09dc] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80c3e70] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x80b2534] /lib32/libpthread.so.0(+0x5e72)[0xf778ae72] /lib32/libc.so.6(clone+0x5e)[0xf75c6d0e] Exiting... Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:44:46 (435): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy 2048 SIGSEGV: segmentation violation Stack trace (10 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf779c400] /lib32/libc.so.6(__printf_fp+0xa83)[0xf7509ee3] /lib32/libc.so.6(_IO_vfprintf+0x2e98)[0xf7506da8] /lib32/libc.so.6(_IO_fprintf+0x1f)[0xf750e1cf] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805060a] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf74dde37] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 18:48:22	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	829,440	1,322,186	1.5941
25 Jul 2011 18:09:24	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	803,520	1,280,887	1.5941
25 Jul 2011 17:41:27	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	777,600	1,239,398	1.5939
25 Jul 2011 17:18:39	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	751,680	1,197,922	1.5937
25 Jul 2011 16:30:26	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	725,760	1,156,772	1.5939
25 Jul 2011 16:04:56	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	699,840	1,115,610	1.5941
25 Jul 2011 15:46:52	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	673,920	1,074,436	1.5943
25 Jul 2011 15:06:07	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	648,000	1,032,975	1.5941
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	622,080	991,581	1.5940
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	596,160	950,275	1.5940
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	570,240	908,829	1.5938
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	544,320	867,581	1.5939
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	518,400	826,234	1.5938
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	492,480	785,078	1.5941
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	466,560	743,623	1.5938
25 Jul 2011 14:41:12	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	440,640	702,259	1.5937
11 Jul 2011 02:41:03	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	414,720	660,915	1.5936
10 Jul 2011 12:28:16	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	388,800	619,528	1.5934
09 Jul 2011 22:52:50	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	362,880	578,016	1.5929
09 Jul 2011 08:35:18	1153762	13028185	hadcm3n_t7fe_1940_40_007317129_1	336,960	536,531	1.5923