Task 11464400

Name	famous_u7m9_599_200_006642716_1
Workunit	6846088
Created	10 Jun 2010, 12:27:07 UTC
Sent	2 Aug 2010, 18:00:20 UTC
Report deadline	2 Nov 2010, 1:27:31 UTC
Received	7 Aug 2010, 10:24:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	786733
Run time	1 days 22 hours 27 min 37 sec
CPU time	1 days 20 hours 11 min 36 sec
Validate state	Invalid
Credit	1,050.06
Device peak FLOPS	3.10 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.36</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77d8400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8137f1a] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8128d21] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf762cbb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5355, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7786400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82dbe6a] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82b0e6a] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832b7ba] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75dabb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7344, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (16 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77a9400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81ad186] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82885cc] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82880dd] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8282d14] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x80832bf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x80a6a3e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x809cf5f] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8078798] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75fdbb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7344, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (12 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7756400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x822a1e2] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82f016e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054c65] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75aabb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7344, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7758400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x80ef89c] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x812a9b0] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75acbb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7344, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (3370): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Aug 2010 09:30:42	786733	11464400	famous_u7m9_599_200_006642716_1	318,266	157,776	0.4957
05 Aug 2010 21:57:35	786733	11464400	famous_u7m9_599_200_006642716_1	308,906	153,276	0.4962
05 Aug 2010 20:39:56	786733	11464400	famous_u7m9_599_200_006642716_1	299,546	148,785	0.4967
05 Aug 2010 19:24:38	786733	11464400	famous_u7m9_599_200_006642716_1	290,186	144,106	0.4966
05 Aug 2010 18:03:31	786733	11464400	famous_u7m9_599_200_006642716_1	280,826	139,628	0.4972
05 Aug 2010 16:47:56	786733	11464400	famous_u7m9_599_200_006642716_1	271,466	134,949	0.4971
05 Aug 2010 15:33:28	786733	11464400	famous_u7m9_599_200_006642716_1	262,106	130,477	0.4978
05 Aug 2010 14:16:28	786733	11464400	famous_u7m9_599_200_006642716_1	252,746	125,994	0.4985
05 Aug 2010 13:00:57	786733	11464400	famous_u7m9_599_200_006642716_1	243,386	121,509	0.4992
05 Aug 2010 03:46:05	786733	11464400	famous_u7m9_599_200_006642716_1	234,026	115,854	0.4950
05 Aug 2010 03:27:33	786733	11464400	famous_u7m9_599_200_006642716_1	224,666	111,351	0.4956
05 Aug 2010 00:36:04	786733	11464400	famous_u7m9_599_200_006642716_1	215,306	106,562	0.4949
04 Aug 2010 20:54:09	786733	11464400	famous_u7m9_599_200_006642716_1	205,946	101,717	0.4939
04 Aug 2010 19:42:13	786733	11464400	famous_u7m9_599_200_006642716_1	196,586	97,206	0.4945
04 Aug 2010 18:57:13	786733	11464400	famous_u7m9_599_200_006642716_1	187,226	92,699	0.4951
04 Aug 2010 17:02:30	786733	11464400	famous_u7m9_599_200_006642716_1	177,866	88,196	0.4959
04 Aug 2010 15:44:05	786733	11464400	famous_u7m9_599_200_006642716_1	168,506	83,694	0.4967
04 Aug 2010 15:32:47	786733	11464400	famous_u7m9_599_200_006642716_1	159,146	79,186	0.4976
04 Aug 2010 13:09:05	786733	11464400	famous_u7m9_599_200_006642716_1	149,786	74,684	0.4986
04 Aug 2010 11:40:16	786733	11464400	famous_u7m9_599_200_006642716_1	140,426	69,596	0.4956