Task 11449347

Name	famous_u5ao_1599_200_006639707_1
Workunit	6843079
Created	10 Jun 2010, 12:00:54 UTC
Sent	26 Jul 2010, 8:32:15 UTC
Report deadline	25 Oct 2010, 15:59:26 UTC
Received	10 Aug 2010, 13:06:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1061362
Run time
CPU time	4 days 10 hours 37 min 14 sec
Validate state	Invalid
Credit	2,872.08
Device peak FLOPS	3.01 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Signal 15 received, exiting... (4520): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (3173): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... (2676): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7498ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2575): called boinc_finish Signal 15 received, exiting... (2841): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2564): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7479ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2556): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... (2525): called boinc_finish Signal 15 received, exiting... (3132): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Signal 15 received, exiting... (2471): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Signal 15 received, exiting... (2460): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf74c8ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2416): called boinc_finish Signal 15 received, exiting... (2832): called boinc_finish Signal 15 received, exiting... (2732): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Signal 15 received, exiting... (2410): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7507ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2450): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2492): called boinc_finish Signal 15 received, exiting... (2794): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2586): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2453): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf743dace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2478): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (2661): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Aug 2010 12:37:01	1061362	11449347	famous_u5ao_1599_200_006639707_1	870,506	382,316	0.4392
10 Aug 2010 11:22:20	1061362	11449347	famous_u5ao_1599_200_006639707_1	861,146	378,273	0.4393
10 Aug 2010 10:21:50	1061362	11449347	famous_u5ao_1599_200_006639707_1	851,786	373,997	0.4391
10 Aug 2010 10:21:50	1061362	11449347	famous_u5ao_1599_200_006639707_1	842,426	369,927	0.4391
10 Aug 2010 10:21:50	1061362	11449347	famous_u5ao_1599_200_006639707_1	833,066	365,851	0.4392
09 Aug 2010 16:15:31	1061362	11449347	famous_u5ao_1599_200_006639707_1	823,706	361,783	0.4392
09 Aug 2010 14:44:07	1061362	11449347	famous_u5ao_1599_200_006639707_1	814,346	357,688	0.4392
09 Aug 2010 13:34:07	1061362	11449347	famous_u5ao_1599_200_006639707_1	804,986	353,603	0.4393
09 Aug 2010 12:28:48	1061362	11449347	famous_u5ao_1599_200_006639707_1	795,626	349,519	0.4393
09 Aug 2010 11:17:07	1061362	11449347	famous_u5ao_1599_200_006639707_1	786,266	345,433	0.4393
09 Aug 2010 10:06:46	1061362	11449347	famous_u5ao_1599_200_006639707_1	776,906	341,348	0.4394
08 Aug 2010 18:22:11	1061362	11449347	famous_u5ao_1599_200_006639707_1	767,546	337,264	0.4394
08 Aug 2010 17:13:52	1061362	11449347	famous_u5ao_1599_200_006639707_1	758,186	333,205	0.4395
08 Aug 2010 15:50:51	1061362	11449347	famous_u5ao_1599_200_006639707_1	748,826	329,032	0.4394
08 Aug 2010 14:41:34	1061362	11449347	famous_u5ao_1599_200_006639707_1	739,466	324,977	0.4395
08 Aug 2010 13:38:29	1061362	11449347	famous_u5ao_1599_200_006639707_1	730,106	320,926	0.4396
08 Aug 2010 12:25:50	1061362	11449347	famous_u5ao_1599_200_006639707_1	720,746	316,867	0.4396
07 Aug 2010 19:09:30	1061362	11449347	famous_u5ao_1599_200_006639707_1	711,386	312,436	0.4392
07 Aug 2010 18:00:27	1061362	11449347	famous_u5ao_1599_200_006639707_1	702,026	308,355	0.4392
06 Aug 2010 20:52:05	1061362	11449347	famous_u5ao_1599_200_006639707_1	692,666	304,283	0.4393