Task 12610763

Name	famous_v4gl_1599_200_006737473_4
Workunit	6940817
Created	21 Feb 2011, 22:12:32 UTC
Sent	21 Feb 2011, 22:12:35 UTC
Report deadline	24 May 2011, 5:39:46 UTC
Received	8 Mar 2011, 10:06:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1135234
Run time
CPU time	21 hours 27 min 40 sec
Validate state	Invalid
Credit	1,297.11
Device peak FLOPS	1.64 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (28408): called boinc_finish Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... unknown signal 30 Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... unknown signal 30 Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... unknown signal 30 Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... Signal 15 received, exiting... unknown signal 30 Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... unknown signal 30 Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... unknown signal 30 Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... SIGSEGV: segmentation violation Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... unknown signal 30 Signal 15 received, exiting... (28406): called boinc_finish (28406): called boinc_finish SIGSEGV: segmentation violation SIGSEGV: segmentation violation unknown signal 30 SIGSEGV: segmentation violation SIGSEGV: segmentation violation SIGSEGV: segmentation violation unknown signal 0 SIGSEGV: segmentation violation unknown signal 0 SIGSEGV: segmentation violation unknown signal 0 Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... Signal 15 received, exiting... (17783): called boinc_finish (17507): called boinc_finish (14315): called boinc_finish (14673): called boinc_finish (15002): called boinc_finish (15328): called boinc_finish (15850): called boinc_finish (16145): called boinc_finish (16478): called boinc_finish (16809): called boinc_finish (17146): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1617): called boinc_finish (1577): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting (13370): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (18944): No heartbeat from core client for 30 sec - exiting (92028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (96709): No heartbeat from core client for 30 sec - exiting (96709): No heartbeat from core client for 30 sec - exiting (96709): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (96779): No heartbeat from core client for 30 sec - exiting (96779): No heartbeat from core client for 30 sec - exiting (96779): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (96799): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (97471): called boinc_finish Signal 15 received, exiting... (96801): called boinc_finish Signal 15 received, exiting... (96781): called boinc_finish Signal 15 received, exiting... (96711): called boinc_finish Signal 15 received, exiting... (92944): called boinc_finish Signal 15 received, exiting... (92723): called boinc_finish Signal 15 received, exiting... (80625): called boinc_finish Signal 15 received, exiting... (8862): called boinc_finish Signal 15 received, exiting... (18947): called boinc_finish Signal 15 received, exiting... (14382): called boinc_finish Signal 15 received, exiting... (1708): called boinc_finish Signal 15 received, exiting... (93620): called boinc_finish Signal 15 received, exiting... (96841): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1621): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (10023): called boinc_finish Signal 15 received, exiting... (9381): called boinc_finish Signal 15 received, exiting... (9057): called boinc_finish Signal 15 received, exiting... (8820): called boinc_finish Signal 15 received, exiting... (8373): called boinc_finish Signal 15 received, exiting... (7891): called boinc_finish Signal 15 received, exiting... (7476): called boinc_finish Signal 15 received, exiting... (7145): called boinc_finish Signal 15 received, exiting... (6296): called boinc_finish Signal 15 received, exiting... (5955): called boinc_finish Signal 15 received, exiting... (5579): called boinc_finish Signal 15 received, exiting... (5235): called boinc_finish Signal 15 received, exiting... (4902): called boinc_finish Signal 15 received, exiting... (4616): called boinc_finish Signal 15 received, exiting... (4213): called boinc_finish Signal 15 received, exiting... (3884): called boinc_finish Signal 15 received, exiting... (3280): called boinc_finish Signal 15 received, exiting... (2944): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... (2500): called boinc_finish (6796): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (5587): called boinc_finish Signal 15 received, exiting... (5252): called boinc_finish Signal 15 received, exiting... (4953): called boinc_finish Signal 15 received, exiting... (4742): called boinc_finish Signal 15 received, exiting... (4409): called boinc_finish Signal 15 received, exiting... (3807): called boinc_finish Signal 15 received, exiting... (3473): called boinc_finish Signal 15 received, exiting... (3146): called boinc_finish Signal 15 received, exiting... (2768): called boinc_finish Signal 15 received, exiting... (2193): called boinc_finish Signal 15 received, exiting... (1709): called boinc_finish Signal 15 received, exiting... (2422): called boinc_finish Signal 15 received, exiting... (5938): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... (1556): called boinc_finish (1716): called boinc_finish Signal 15 received, exiting... (1556): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... (1619): called boinc_finish (1630): called boinc_finish Signal 15 received, exiting... (1619): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1727): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1626): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (1591): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (5908): called boinc_finish Signal 15 received, exiting... (5508): called boinc_finish Signal 15 received, exiting... (5297): called boinc_finish Signal 15 received, exiting... (4962): called boinc_finish Signal 15 received, exiting... (4752): called boinc_finish Signal 15 received, exiting... (4519): called boinc_finish Signal 15 received, exiting... (4176): called boinc_finish Signal 15 received, exiting... (3576): called boinc_finish Signal 15 received, exiting... (3238): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (13599): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	393,146	72,651	0.1848
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	383,786	82,893	0.2160
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	374,426	74,786	0.1997
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	365,066	75,905	0.2079
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	355,706	73,722	0.2073
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	346,346	68,200	0.1969
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	336,986	78,814	0.2339
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	327,626	66,744	0.2037
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	318,266	74,426	0.2338
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	308,906	72,248	0.2339
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	299,546	70,037	0.2338
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	290,186	64,524	0.2224
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	280,826	68,151	0.2427
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	271,466	62,664	0.2308
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	262,106	60,439	0.2306
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	252,746	51,588	0.2041
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	243,386	46,076	0.1893
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	234,026	36,111	0.1543
08 Mar 2011 10:11:52	1135234	12610763	famous_v4gl_1599_200_006737473_4	224,666	48,298	0.2150
08 Mar 2011 10:11:51	1135234	12610763	famous_v4gl_1599_200_006737473_4	215,306	39,464	0.1833