Task 12477102

Name	famous_x11m_1599_200_007063185_1
Workunit	7266485
Created	12 Jan 2011, 19:00:00 UTC
Sent	12 Jan 2011, 19:03:04 UTC
Report deadline	14 Apr 2011, 2:30:15 UTC
Received	8 Mar 2011, 12:28:21 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	944830
Run time
CPU time	5 days 13 hours 52 min 52 sec
Validate state	Invalid
Credit	2,223.56
Device peak FLOPS	2.49 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.2.14</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> for 30 sec - exiting (27249): No heartbeat from core client for 30 sec - exiting (27249): No heartbeat from core client for 30 sec - exiting (27249): No heartbeat from core client for 30 sec - exiting (27249): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27322): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27405): No heartbeat from core client for 30 sec - exiting (27500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting (27631): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27631): No heartbeat from core client for 30 sec - exiting (27678): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27678): No heartbeat from core client for 30 sec - exiting (27678): No heartbeat from core client for 30 sec - exiting (27742): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27742): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27826): No heartbeat from core client for 30 sec - exiting (27971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (27971): No heartbeat from core client for 30 sec - exiting (27971): No heartbeat from core client for 30 sec - exiting (27971): No heartbeat from core client for 30 sec - exiting (27971): No heartbeat from core client for 30 sec - exiting (27971): No heartbeat from core client for 30 sec - exiting (27971): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28087): No heartbeat from core client for 30 sec - exiting (28087): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28149): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting (28149): No heartbeat from core client for 30 sec - exiting (28211): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy (28276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28333): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28333): No heartbeat from core client for 30 sec - exiting (28436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28436): No heartbeat from core client for 30 sec - exiting (28436): No heartbeat from core client for 30 sec - exiting (28436): No heartbeat from core client for 30 sec - exiting (28436): No heartbeat from core client for 30 sec - exiting (28436): No heartbeat from core client for 30 sec - exiting (28436): No heartbeat from core client for 30 sec - exiting (28520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28520): No heartbeat from core client for 30 sec - exiting (28520): No heartbeat from core client for 30 sec - exiting (28520): No heartbeat from core client for 30 sec - exiting (28520): No heartbeat from core client for 30 sec - exiting (28520): No heartbeat from core client for 30 sec - exiting (28520): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting (28587): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28653): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28701): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28701): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28771): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting (28879): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28879): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting (28945): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28945): No heartbeat from core client for 30 sec - exiting (29015): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (29015): No heartbeat from core client for 30 sec - exiting (29015): No heartbeat from core client for 30 sec - exiting (29015): No heartbeat from core client for 30 sec - exiting (29015): No heartbeat from core client for 30 sec - exiting (29015): No heartbeat from core client for 30 sec - exiting (29015): No heartbeat from core client for 30 sec - exiting (29080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (29080): No heartbeat from core client for 30 sec - exiting (29080): No heartbeat from core client for 30 sec - exiting (29080): No heartbeat from core client for 30 sec - exiting (29080): No heartbeat from core client for 30 sec - exiting (29080): No heartbeat from core client for 30 sec - exiting (29080): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29167): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting (29254): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=25741, selfPID=25741, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting (26703): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5103, selfPID=5103, iMonCtr=1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (18148): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Feb 2011 01:56:38	944830	12477102	famous_x11m_1599_200_007063185_1	673,946	477,419	0.7084
28 Feb 2011 00:17:12	944830	12477102	famous_x11m_1599_200_007063185_1	664,586	472,389	0.7108
27 Feb 2011 22:32:15	944830	12477102	famous_x11m_1599_200_007063185_1	655,226	467,249	0.7131
27 Feb 2011 20:51:20	944830	12477102	famous_x11m_1599_200_007063185_1	645,866	461,941	0.7152
27 Feb 2011 15:13:52	944830	12477102	famous_x11m_1599_200_007063185_1	636,506	456,512	0.7172
27 Feb 2011 15:13:53	944830	12477102	famous_x11m_1599_200_007063185_1	627,146	451,447	0.7198
27 Feb 2011 15:13:47	944830	12477102	famous_x11m_1599_200_007063185_1	617,786	446,400	0.7226
27 Feb 2011 15:13:53	944830	12477102	famous_x11m_1599_200_007063185_1	608,426	441,376	0.7254
27 Feb 2011 00:27:51	944830	12477102	famous_x11m_1599_200_007063185_1	599,066	436,348	0.7284
26 Feb 2011 22:57:10	944830	12477102	famous_x11m_1599_200_007063185_1	589,706	431,327	0.7314
26 Feb 2011 21:09:45	944830	12477102	famous_x11m_1599_200_007063185_1	580,346	426,046	0.7341
26 Feb 2011 19:18:25	944830	12477102	famous_x11m_1599_200_007063185_1	570,986	420,980	0.7373
26 Feb 2011 05:56:41	944830	12477102	famous_x11m_1599_200_007063185_1	561,626	415,659	0.7401
26 Feb 2011 03:53:52	944830	12477102	famous_x11m_1599_200_007063185_1	552,266	410,260	0.7429
26 Feb 2011 01:53:38	944830	12477102	famous_x11m_1599_200_007063185_1	542,906	405,036	0.7461
25 Feb 2011 23:46:08	944830	12477102	famous_x11m_1599_200_007063185_1	533,546	399,684	0.7491
25 Feb 2011 19:28:55	944830	12477102	famous_x11m_1599_200_007063185_1	524,186	394,404	0.7524
25 Feb 2011 15:08:27	944830	12477102	famous_x11m_1599_200_007063185_1	514,826	389,270	0.7561
25 Feb 2011 14:01:43	944830	12477102	famous_x11m_1599_200_007063185_1	505,466	384,224	0.7601
24 Feb 2011 23:23:15	944830	12477102	famous_x11m_1599_200_007063185_1	496,106	379,116	0.7642