Task 11883466

Name	famous_vpd9_1599_200_006718643_5
Workunit	6921896
Created	13 Sep 2010, 4:57:06 UTC
Sent	13 Sep 2010, 5:00:26 UTC
Report deadline	13 Dec 2010, 12:27:37 UTC
Received	13 Sep 2010, 23:21:39 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	904604
Run time
CPU time	3 hours 5 min 22 sec
Validate state	Invalid
Credit	216.26
Device peak FLOPS	2.16 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-apple-darwin
Stderr	<core_client_version>6.2.18</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> (23065): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23153): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23153): No heartbeat from core client for 30 sec - exiting (23153): No heartbeat from core client for 30 sec - exiting (23153): No heartbeat from core client for 30 sec - exiting (23153): No heartbeat from core client for 30 sec - exiting (23153): No heartbeat from core client for 30 sec - exiting (23153): No heartbeat from core client for 30 sec - exiting (23153): No heartbeat from core client for 30 sec - exiting (23442): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23442): No heartbeat from core client for 30 sec - exiting (23494): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23494): No heartbeat from core client for 30 sec - exiting (23494): No heartbeat from core client for 30 sec - exiting (23494): No heartbeat from core client for 30 sec - exiting (23494): No heartbeat from core client for 30 sec - exiting (23494): No heartbeat from core client for 30 sec - exiting (23494): No heartbeat from core client for 30 sec - exiting (23494): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23725): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting (23725): No heartbeat from core client for 30 sec - exiting (23782): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23782): No heartbeat from core client for 30 sec - exiting (23782): No heartbeat from core client for 30 sec - exiting (23782): No heartbeat from core client for 30 sec - exiting (23796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23897): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23897): No heartbeat from core client for 30 sec - exiting (23922): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23966): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (23988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24086): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24086): No heartbeat from core client for 30 sec - exiting (24115): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24115): No heartbeat from core client for 30 sec - exiting (24115): No heartbeat from core client for 30 sec - exiting (24115): No heartbeat from core client for 30 sec - exiting (24115): No heartbeat from core client for 30 sec - exiting (24115): No heartbeat from core client for 30 sec - exiting (24115): No heartbeat from core client for 30 sec - exiting (24142): No heartbeat from core client for 30 sec - exiting (24142): No heartbeat from core client for 30 sec - exiting (24142): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting (24198): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting (24211): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24211): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (24221): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24221): No heartbeat from core client for 30 sec - exiting (24221): No heartbeat from core client for 30 sec - exiting (24221): No heartbeat from core client for 30 sec - exiting (24221): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24249): No heartbeat from core client for 30 sec - exiting (24266): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24287): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24287): No heartbeat from core client for 30 sec - exiting (24303): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24323): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24323): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24411): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24448): No heartbeat from core client for 30 sec - exiting (24468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24488): No heartbeat from core client for 30 sec - exiting (24506): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24506): No heartbeat from core client for 30 sec - exiting (24506): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24572): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting (24572): No heartbeat from core client for 30 sec - exiting (24588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24622): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24622): No heartbeat from core client for 30 sec - exiting (24622): No heartbeat from core client for 30 sec - exiting (24622): No heartbeat from core client for 30 sec - exiting (24622): No heartbeat from core client for 30 sec - exiting (24622): No heartbeat from core client for 30 sec - exiting (24622): No heartbeat from core client for 30 sec - exiting (24622): No heartbeat from core client for 30 sec - exiting (24622): No heartbeat from core client for 30 sec - exiting (24701): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (24769): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24769): No heartbeat from core client for 30 sec - exiting (24793): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24793): No heartbeat from core client for 30 sec - exiting (24793): No heartbeat from core client for 30 sec - exiting (24793): No heartbeat from core client for 30 sec - exiting (24793): No heartbeat from core client for 30 sec - exiting (24793): No heartbeat from core client for 30 sec - exiting (24793): No heartbeat from core client for 30 sec - exiting (24793): No heartbeat from core client for 30 sec - exiting (24806): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24806): No heartbeat from core client for 30 sec - exiting (24902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (24967): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25037): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25098): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25098): No heartbeat from core client for 30 sec - exiting (25098): No heartbeat from core client for 30 sec - exiting (25098): No heartbeat from core client for 30 sec - exiting (25420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25443): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25459): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25459): No heartbeat from core client for 30 sec - exiting (25459): No heartbeat from core client for 30 sec - exiting (25459): No heartbeat from core client for 30 sec - exiting (25492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25520): No heartbeat from core client for 30 sec - exiting (25741): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (25741): No heartbeat from core client for 30 sec - exiting (25741): No heartbeat from core client for 30 sec - exiting (25741): No heartbeat from core client for 30 sec - exiting (25751): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy Sorry, too many model crashes! :-( (25827): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Sep 2010 23:27:40	904604	11883466	famous_vpd9_1599_200_006718643_5	65,546	10,775	0.1644
13 Sep 2010 23:27:40	904604	11883466	famous_vpd9_1599_200_006718643_5	56,186	9,236	0.1644
13 Sep 2010 10:54:53	904604	11883466	famous_vpd9_1599_200_006718643_5	46,826	7,686	0.1641
13 Sep 2010 09:44:33	904604	11883466	famous_vpd9_1599_200_006718643_5	37,466	6,011	0.1604
13 Sep 2010 08:25:04	904604	11883466	famous_vpd9_1599_200_006718643_5	28,106	4,484	0.1595
13 Sep 2010 07:10:26	904604	11883466	famous_vpd9_1599_200_006718643_5	18,746	3,329	0.1776
13 Sep 2010 06:11:29	904604	11883466	famous_vpd9_1599_200_006718643_5	9,386	1,690	0.1801