Task 11606450

Name	famous_r171_999_200_006632680_6
Workunit	6836052
Created	14 Jul 2010, 15:10:07 UTC
Sent	14 Jul 2010, 18:25:39 UTC
Report deadline	26 Jun 2011, 23:45:39 UTC
Received	26 Aug 2010, 1:01:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1027631
Run time
CPU time	7 days 22 hours 32 min 51 sec
Validate state	Invalid
Credit	1,297.11
Device peak FLOPS	0.25 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... (9184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (2150): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1615, selfPID=1615, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5693, selfPID=5693, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (1920): No heartbeat from core client for 30 sec - exiting (1920): No heartbeat from core client for 30 sec - exiting (1920): No heartbeat from core client for 30 sec - exiting (1920): No heartbeat from core client for 30 sec - exiting (4164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8073): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8073): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting (8777): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8820): No heartbeat from core client for 30 sec - exiting (8848): No heartbeat from core client for 30 sec - exiting (8848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting (8895): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8895): No heartbeat from core client for 30 sec - exiting (8900): No heartbeat from core client for 30 sec - exiting (8900): No heartbeat from core client for 30 sec - exiting (8900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8915): No heartbeat from core client for 30 sec - exiting (8915): No heartbeat from core client for 30 sec - exiting (8915): No heartbeat from core client for 30 sec - exiting (8915): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8919): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (8919): No heartbeat from core client for 30 sec - exiting (9117): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (9826): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (15666): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Aug 2010 21:59:32	1027631	11606450	famous_r171_999_200_006632680_6	393,146	676,207	1.7200
25 Aug 2010 17:38:00	1027631	11606450	famous_r171_999_200_006632680_6	383,786	664,315	1.7310
25 Aug 2010 13:53:19	1027631	11606450	famous_r171_999_200_006632680_6	374,426	653,505	1.7454
24 Aug 2010 18:37:10	1027631	11606450	famous_r171_999_200_006632680_6	365,066	639,111	1.7507
24 Aug 2010 03:11:58	1027631	11606450	famous_r171_999_200_006632680_6	355,706	624,008	1.7543
23 Aug 2010 16:43:38	1027631	11606450	famous_r171_999_200_006632680_6	346,346	612,755	1.7692
23 Aug 2010 01:54:37	1027631	11606450	famous_r171_999_200_006632680_6	336,986	600,289	1.7813
19 Aug 2010 20:18:20	1027631	11606450	famous_r171_999_200_006632680_6	327,626	581,496	1.7749
19 Aug 2010 13:21:26	1027631	11606450	famous_r171_999_200_006632680_6	318,266	563,821	1.7715
19 Aug 2010 05:48:12	1027631	11606450	famous_r171_999_200_006632680_6	308,906	537,409	1.7397
18 Aug 2010 22:00:44	1027631	11606450	famous_r171_999_200_006632680_6	299,546	511,092	1.7062
18 Aug 2010 16:00:19	1027631	11606450	famous_r171_999_200_006632680_6	290,186	492,710	1.6979
17 Aug 2010 21:06:30	1027631	11606450	famous_r171_999_200_006632680_6	280,826	477,144	1.6991
17 Aug 2010 15:17:16	1027631	11606450	famous_r171_999_200_006632680_6	271,466	459,884	1.6941
16 Aug 2010 22:41:55	1027631	11606450	famous_r171_999_200_006632680_6	262,106	443,646	1.6926
16 Aug 2010 22:41:55	1027631	11606450	famous_r171_999_200_006632680_6	252,746	425,993	1.6855
13 Aug 2010 19:48:51	1027631	11606450	famous_r171_999_200_006632680_6	243,386	415,562	1.7074
12 Aug 2010 22:53:57	1027631	11606450	famous_r171_999_200_006632680_6	234,026	400,413	1.7110
12 Aug 2010 13:33:12	1027631	11606450	famous_r171_999_200_006632680_6	224,666	386,070	1.7184
12 Aug 2010 08:24:52	1027631	11606450	famous_r171_999_200_006632680_6	215,306	368,044	1.7094