Task 11842836

Name	famous_vnrk_1799_200_006716566_2
Workunit	6919819
Created	26 Aug 2010, 18:03:43 UTC
Sent	16 Sep 2010, 5:00:08 UTC
Report deadline	16 Dec 2010, 12:27:19 UTC
Received	16 Sep 2010, 23:46:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	904604
Run time	0 sec
CPU time	3 hours 14 min 31 sec
Validate state	Invalid
Credit	216.26
Device peak FLOPS	2.16 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-apple-darwin
Stderr	<core_client_version>6.2.18</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> (15269): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15273): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15273): No heartbeat from core client for 30 sec - exiting (15273): No heartbeat from core client for 30 sec - exiting (15273): No heartbeat from core client for 30 sec - exiting (15315): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15327): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15327): No heartbeat from core client for 30 sec - exiting (15327): No heartbeat from core client for 30 sec - exiting (15327): No heartbeat from core client for 30 sec - exiting (15385): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15385): No heartbeat from core client for 30 sec - exiting (15415): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15415): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15445): No heartbeat from core client for 30 sec - exiting (15470): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15488): No heartbeat from core client for 30 sec - exiting (15526): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15526): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting (15541): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15582): No heartbeat from core client for 30 sec - exiting (15611): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (15683): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15706): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (15737): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15766): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting (15766): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (15901): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (16059): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16082): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16082): No heartbeat from core client for 30 sec - exiting (16088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16088): No heartbeat from core client for 30 sec - exiting (16088): No heartbeat from core client for 30 sec - exiting (16088): No heartbeat from core client for 30 sec - exiting (16088): No heartbeat from core client for 30 sec - exiting (16130): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16185): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (16229): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16229): No heartbeat from core client for 30 sec - exiting (16229): No heartbeat from core client for 30 sec - exiting (16229): No heartbeat from core client for 30 sec - exiting (16229): No heartbeat from core client for 30 sec - exiting (16229): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (16243): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16243): No heartbeat from core client for 30 sec - exiting (16243): No heartbeat from core client for 30 sec - exiting (16243): No heartbeat from core client for 30 sec - exiting (16243): No heartbeat from core client for 30 sec - exiting (16243): No heartbeat from core client for 30 sec - exiting (16302): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16302): No heartbeat from core client for 30 sec - exiting (16302): No heartbeat from core client for 30 sec - exiting (16302): No heartbeat from core client for 30 sec - exiting (16302): No heartbeat from core client for 30 sec - exiting (16302): No heartbeat from core client for 30 sec - exiting (16327): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16364): No heartbeat from core client for 30 sec - exiting (16364): No heartbeat from core client for 30 sec - exiting (16364): No heartbeat from core client for 30 sec - exiting (16364): No heartbeat from core client for 30 sec - exiting (16364): No heartbeat from core client for 30 sec - exiting (16364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... (16586): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16586): No heartbeat from core client for 30 sec - exiting (16608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (16816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16838): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (16850): No heartbeat from core client for 30 sec - exiting (16850): No heartbeat from core client for 30 sec - exiting (16850): No heartbeat from core client for 30 sec - exiting (16850): No heartbeat from core client for 30 sec - exiting (16850): No heartbeat from core client for 30 sec - exiting (16850): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (16883): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting (16895): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 (17194): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17226): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17339): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting (17352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... (17520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (17520): No heartbeat from core client for 30 sec - exiting (17520): No heartbeat from core client for 30 sec - exiting (17520): No heartbeat from core client for 30 sec - exiting (17520): No heartbeat from core client for 30 sec - exiting (17520): No heartbeat from core client for 30 sec - exiting (17520): No heartbeat from core client for 30 sec - exiting (17520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (18421): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (18421): No heartbeat from core client for 30 sec - exiting Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (18438): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Sep 2010 23:49:48	904604	11842836	famous_vnrk_1799_200_006716566_2	65,546	10,694	0.1632
16 Sep 2010 23:49:48	904604	11842836	famous_vnrk_1799_200_006716566_2	56,186	9,212	0.1640
16 Sep 2010 23:49:48	904604	11842836	famous_vnrk_1799_200_006716566_2	46,826	7,628	0.1629
16 Sep 2010 23:49:48	904604	11842836	famous_vnrk_1799_200_006716566_2	37,466	6,009	0.1604
16 Sep 2010 23:49:48	904604	11842836	famous_vnrk_1799_200_006716566_2	28,106	4,508	0.1604
16 Sep 2010 23:49:48	904604	11842836	famous_vnrk_1799_200_006716566_2	18,746	2,886	0.1540
16 Sep 2010 07:12:18	904604	11842836	famous_vnrk_1799_200_006716566_2	9,386	1,417	0.1510