Task 12648195

Name	famous_xfun_2099_200_007082374_2
Workunit	7285674
Created	8 Mar 2011, 11:28:05 UTC
Sent	8 Mar 2011, 11:28:08 UTC
Report deadline	7 Jun 2011, 18:55:19 UTC
Received	26 Mar 2011, 11:09:17 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1061362
Run time
CPU time	6 days 13 hours 34 min 38 sec
Validate state	Invalid
Credit	3,705.88
Device peak FLOPS	2.99 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (9269): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf74ebace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (6422): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf74a2ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy (3316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (27770): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7498ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2442): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf74d0ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Signal 15 received, exiting... (2634): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf74cface] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2502): called boinc_finish Signal 15 received, exiting... (2473): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2649): called boinc_finish Signal 15 received, exiting... (3227): called boinc_finish Signal 15 received, exiting... (2511): called boinc_finish Signal 15 received, exiting... (2448): called boinc_finish Signal 15 received, exiting... (2514): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf74b5ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Signal 15 received, exiting... Signal 15 received, exiting... (2677): called boinc_finish Signal 15 received, exiting... (2471): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7508ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2468): called boinc_finish Signal 15 received, exiting... (2690): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf744eace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2441): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2696): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2474): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7485ace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2953): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Signal 15 received, exiting... (2694): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xffffe400] famous_6.11_i686-pc-linux-gnu[0x804f906] famous_6.11_i686-pc-linux-gnu[0x805085a] famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib/libc.so.6(__libc_start_main+0xfe)[0xf74bbace] famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... Signal 15 received, exiting... (2500): called boinc_finish Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy Sorry, too many model crashes! :-( (2688): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Mar 2011 10:45:49	1061362	12648195	famous_xfun_2099_200_007082374_2	1,123,226	565,928	0.5038
26 Mar 2011 09:27:54	1061362	12648195	famous_xfun_2099_200_007082374_2	1,113,866	561,239	0.5039
25 Mar 2011 21:06:47	1061362	12648195	famous_xfun_2099_200_007082374_2	1,104,506	556,551	0.5039
25 Mar 2011 19:47:01	1061362	12648195	famous_xfun_2099_200_007082374_2	1,095,146	551,872	0.5039
25 Mar 2011 18:25:42	1061362	12648195	famous_xfun_2099_200_007082374_2	1,085,786	547,190	0.5040
25 Mar 2011 17:09:31	1061362	12648195	famous_xfun_2099_200_007082374_2	1,076,426	542,510	0.5040
25 Mar 2011 17:09:31	1061362	12648195	famous_xfun_2099_200_007082374_2	1,067,066	537,824	0.5040
25 Mar 2011 17:09:31	1061362	12648195	famous_xfun_2099_200_007082374_2	1,057,706	533,146	0.5041
25 Mar 2011 17:09:31	1061362	12648195	famous_xfun_2099_200_007082374_2	1,048,346	528,467	0.5041
25 Mar 2011 11:53:00	1061362	12648195	famous_xfun_2099_200_007082374_2	1,038,986	523,785	0.5041
24 Mar 2011 21:24:48	1061362	12648195	famous_xfun_2099_200_007082374_2	1,029,626	519,086	0.5042
24 Mar 2011 20:29:13	1061362	12648195	famous_xfun_2099_200_007082374_2	1,020,266	514,382	0.5042
24 Mar 2011 18:52:01	1061362	12648195	famous_xfun_2099_200_007082374_2	1,010,906	509,676	0.5042
24 Mar 2011 17:14:29	1061362	12648195	famous_xfun_2099_200_007082374_2	1,001,546	504,974	0.5042
24 Mar 2011 15:50:56	1061362	12648195	famous_xfun_2099_200_007082374_2	992,186	500,263	0.5042
24 Mar 2011 14:35:13	1061362	12648195	famous_xfun_2099_200_007082374_2	982,826	495,566	0.5042
24 Mar 2011 13:14:01	1061362	12648195	famous_xfun_2099_200_007082374_2	973,466	490,841	0.5042
24 Mar 2011 11:52:19	1061362	12648195	famous_xfun_2099_200_007082374_2	964,106	486,129	0.5042
24 Mar 2011 09:40:51	1061362	12648195	famous_xfun_2099_200_007082374_2	954,746	481,412	0.5042
24 Mar 2011 08:15:44	1061362	12648195	famous_xfun_2099_200_007082374_2	945,386	476,608	0.5041