Task 13569578

Name	hadcm3n_yj86_1900_40_007514944_4
Workunit	7712419
Created	30 Oct 2011, 20:13:44 UTC
Sent	30 Oct 2011, 20:19:53 UTC
Report deadline	30 Jan 2012, 3:47:04 UTC
Received	30 Nov 2011, 1:37:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	689984
Run time	18 days 8 hours 20 min 41 sec
CPU time	14 days 21 hours 17 min 29 sec
Validate state	Invalid
Credit	10,886.40
Device peak FLOPS	3.03 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> =0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:51:40 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:44:45 (15780): No heartbeat from core client for 30 sec - exiting 18:44:46 (15780): No heartbeat from core client for 30 sec - exiting 18:44:47 (15780): No heartbeat from core client for 30 sec - exiting 18:44:48 (15780): No heartbeat from core client for 30 sec - exiting 18:44:49 (15780): No heartbeat from core client for 30 sec - exiting 18:44:50 (15780): No heartbeat from core client for 30 sec - exiting 18:44:51 (15780): No heartbeat from core client for 30 sec - exiting 18:44:52 (15780): No heartbeat from core client for 30 sec - exiting 18:44:53 (15780): No heartbeat from core client for 30 sec - exiting 18:44:54 (15780): No heartbeat from core client for 30 sec - exiting 18:44:55 (15780): No heartbeat from core client for 30 sec - exiting 18:44:56 (15780): No heartbeat from core client for 30 sec - exiting 18:44:57 (15780): No heartbeat from core client for 30 sec - exiting 18:44:58 (15780): No heartbeat from core client for 30 sec - exiting 18:44:59 (15780): No heartbeat from core client for 30 sec - exiting 18:45:00 (15780): No heartbeat from core client for 30 sec - exiting 18:45:01 (15780): No heartbeat from core client for 30 sec - exiting 18:45:02 (15780): No heartbeat from core client for 30 sec - exiting 18:45:03 (15780): No heartbeat from core client for 30 sec - exiting 18:45:04 (15780): No heartbeat from core client for 30 sec - exiting 18:45:05 (15780): No heartbeat from core client for 30 sec - exiting 18:45:06 (15780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:45:07 (15780): No heartbeat from core client for 30 sec - exiting 18:45:08 (15780): No heartbeat from core client for 30 sec - exiting 18:45:09 (15780): No heartbeat from core client for 30 sec - exiting 18:45:10 (15780): No heartbeat from core client for 30 sec - exiting 18:45:11 (15780): No heartbeat from core client for 30 sec - exiting 18:45:12 (15780): No heartbeat from core client for 30 sec - exiting 18:45:13 (15780): No heartbeat from core client for 30 sec - exiting 18:45:14 (15780): No heartbeat from core client for 30 sec - exiting 18:45:15 (15780): No heartbeat from core client for 30 sec - exiting 18:45:16 (15780): No heartbeat from core client for 30 sec - exiting 18:45:17 (15780): No heartbeat from core client for 30 sec - exiting 18:46:28 (13712): No heartbeat from core client for 30 sec - exiting 18:46:29 (13712): No heartbeat from core client for 30 sec - exiting 18:46:30 (13712): No heartbeat from core client for 30 sec - exiting 18:46:31 (13712): No heartbeat from core client for 30 sec - exiting 18:46:32 (13712): No heartbeat from core client for 30 sec - exiting 18:46:33 (13712): No heartbeat from core client for 30 sec - exiting 18:46:34 (13712): No heartbeat from core client for 30 sec - exiting 18:46:35 (13712): No heartbeat from core client for 30 sec - exiting 18:46:36 (13712): No heartbeat from core client for 30 sec - exiting 18:46:37 (13712): No heartbeat from core client for 30 sec - exiting 18:46:38 (13712): No heartbeat from core client for 30 sec - exiting 18:46:39 (13712): No heartbeat from core client for 30 sec - exiting 18:46:40 (13712): No heartbeat from core client for 30 sec - exiting 18:46:41 (13712): No heartbeat from core client for 30 sec - exiting 18:46:42 (13712): No heartbeat from core client for 30 sec - exiting 18:46:43 (13712): No heartbeat from core client for 30 sec - exiting 18:46:44 (13712): No heartbeat from core client for 30 sec - exiting 18:46:45 (13712): No heartbeat from core client for 30 sec - exiting 18:46:46 (13712): No heartbeat from core client for 30 sec - exiting 18:46:47 (13712): No heartbeat from core client for 30 sec - exiting 18:46:48 (13712): No heartbeat from core client for 30 sec - exiting 18:46:50 (13712): No heartbeat from core client for 30 sec - exiting 18:46:51 (13712): No heartbeat from core client for 30 sec - exiting 18:46:52 (13712): No heartbeat from core client for 30 sec - exiting 18:46:53 (13712): No heartbeat from core client for 30 sec - exiting 18:46:54 (13712): No heartbeat from core client for 30 sec - exiting 18:46:55 (13712): No heartbeat from core client for 30 sec - exiting 18:46:56 (13712): No heartbeat from core client for 30 sec - exiting 18:46:57 (13712): No heartbeat from core client for 30 sec - exiting 18:46:58 (13712): No heartbeat from core client for 30 sec - exiting 18:46:59 (13712): No heartbeat from core client for 30 sec - exiting 18:47:00 (13712): No heartbeat from core client for 30 sec - exiting 18:47:01 (13712): No heartbeat from core client for 30 sec - exiting 18:47:02 (13712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:48:44 (376): No heartbeat from core client for 30 sec - exiting 18:48:46 (376): No heartbeat from core client for 30 sec - exiting 18:48:47 (376): No heartbeat from core client for 30 sec - exiting 18:48:48 (376): No heartbeat from core client for 30 sec - exiting 18:48:49 (376): No heartbeat from core client for 30 sec - exiting 18:48:50 (376): No heartbeat from core client for 30 sec - exiting 18:48:51 (376): No heartbeat from core client for 30 sec - exiting 18:48:52 (376): No heartbeat from core client for 30 sec - exiting 18:48:53 (376): No heartbeat from core client for 30 sec - exiting 18:48:54 (376): No heartbeat from core client for 30 sec - exiting 18:48:55 (376): No heartbeat from core client for 30 sec - exiting 18:48:56 (376): No heartbeat from core client for 30 sec - exiting 18:48:57 (376): No heartbeat from core client for 30 sec - exiting 18:48:58 (376): No heartbeat from core client for 30 sec - exiting 18:48:59 (376): No heartbeat from core client for 30 sec - exiting 18:49:00 (376): No heartbeat from core client for 30 sec - exiting 18:49:01 (376): No heartbeat from core client for 30 sec - exiting 18:49:02 (376): No heartbeat from core client for 30 sec - exiting 18:49:03 (376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:04 (376): No heartbeat from core client for 30 sec - exiting 18:49:05 (376): No heartbeat from core client for 30 sec - exiting 18:49:06 (376): No heartbeat from core client for 30 sec - exiting 18:49:07 (376): No heartbeat from core client for 30 sec - exiting 18:49:08 (376): No heartbeat from core client for 30 sec - exiting 18:49:09 (376): No heartbeat from core client for 30 sec - exiting 18:49:10 (376): No heartbeat from core client for 30 sec - exiting 18:49:11 (376): No heartbeat from core client for 30 sec - exiting 18:49:12 (376): No heartbeat from core client for 30 sec - exiting 18:49:13 (376): No heartbeat from core client for 30 sec - exiting 18:55:06 (13516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:40 (14660): No heartbeat from core client for 30 sec - exiting 18:56:41 (14660): No heartbeat from core client for 30 sec - exiting 18:56:42 (14660): No heartbeat from core client for 30 sec - exiting 18:56:43 (14660): No heartbeat from core client for 30 sec - exiting 18:56:44 (14660): No heartbeat from core client for 30 sec - exiting 18:56:45 (14660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:46 (14660): No heartbeat from core client for 30 sec - exiting 18:56:47 (14660): No heartbeat from core client for 30 sec - exiting 18:56:48 (14660): No heartbeat from core client for 30 sec - exiting 18:56:49 (14660): No heartbeat from core client for 30 sec - exiting 18:56:50 (14660): No heartbeat from core client for 30 sec - exiting 18:56:51 (14660): No heartbeat from core client for 30 sec - exiting 18:56:52 (14660): No heartbeat from core client for 30 sec - exiting 18:56:53 (14660): No heartbeat from core client for 30 sec - exiting 18:56:54 (14660): No heartbeat from core client for 30 sec - exiting 18:57:55 (13872): No heartbeat from core client for 30 sec - exiting 18:57:57 (13872): No heartbeat from core client for 30 sec - exiting 18:57:58 (13872): No heartbeat from core client for 30 sec - exiting 18:57:59 (13872): No heartbeat from core client for 30 sec - exiting 18:58:00 (13872): No heartbeat from core client for 30 sec - exiting 18:58:01 (13872): No heartbeat from core client for 30 sec - exiting 18:58:02 (13872): No heartbeat from core client for 30 sec - exiting 18:58:03 (13872): No heartbeat from core client for 30 sec - exiting 18:58:04 (13872): No heartbeat from core client for 30 sec - exiting 18:58:35 (13872): No heartbeat from core client for 30 sec - exiting 18:58:37 (13872): No heartbeat from core client for 30 sec - exiting 18:58:38 (13872): No heartbeat from core client for 30 sec - exiting 18:58:39 (13872): No heartbeat from core client for 30 sec - exiting 18:58:40 (13872): No heartbeat from core client for 30 sec - exiting 18:58:41 (13872): No heartbeat from core client for 30 sec - exiting 18:58:42 (13872): No heartbeat from core client for 30 sec - exiting 18:58:43 (13872): No heartbeat from core client for 30 sec - exiting 18:58:44 (13872): No heartbeat from core client for 30 sec - exiting 18:58:45 (13872): No heartbeat from core client for 30 sec - exiting 18:58:46 (13872): No heartbeat from core client for 30 sec - exiting 18:58:47 (13872): No heartbeat from core client for 30 sec - exiting 18:58:48 (13872): No heartbeat from core client for 30 sec - exiting 18:58:49 (13872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:50 (13872): No heartbeat from core client for 30 sec - exiting 18:58:51 (13872): No heartbeat from core client for 30 sec - exiting 18:58:52 (13872): No heartbeat from core client for 30 sec - exiting 18:58:53 (13872): No heartbeat from core client for 30 sec - exiting 18:58:54 (13872): No heartbeat from core client for 30 sec - exiting 18:58:55 (13872): No heartbeat from core client for 30 sec - exiting 18:58:56 (13872): No heartbeat from core client for 30 sec - exiting 18:58:57 (13872): No heartbeat from core client for 30 sec - exiting 18:58:58 (13872): No heartbeat from core client for 30 sec - exiting 18:58:59 (13872): No heartbeat from core client for 30 sec - exiting 19:02:54 (5120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:04:07 (4592): No heartbeat from core client for 30 sec - exiting 19:04:08 (4592): No heartbeat from core client for 30 sec - exiting 19:04:09 (4592): No heartbeat from core client for 30 sec - exiting 19:04:10 (4592): No heartbeat from core client for 30 sec - exiting 19:04:11 (4592): No heartbeat from core client for 30 sec - exiting 19:04:12 (4592): No heartbeat from core client for 30 sec - exiting 19:04:13 (4592): No heartbeat from core client for 30 sec - exiting 19:04:14 (4592): No heartbeat from core client for 30 sec - exiting 19:04:15 (4592): No heartbeat from core client for 30 sec - exiting 19:04:16 (4592): No heartbeat from core client for 30 sec - exiting 19:04:17 (4592): No heartbeat from core client for 30 sec - exiting 19:04:18 (4592): No heartbeat from core client for 30 sec - exiting 19:04:19 (4592): No heartbeat from core client for 30 sec - exiting 19:04:20 (4592): No heartbeat from core client for 30 sec - exiting 19:04:21 (4592): No heartbeat from core client for 30 sec - exiting 19:04:22 (4592): No heartbeat from core client for 30 sec - exiting 19:04:23 (4592): No heartbeat from core client for 30 sec - exiting 19:04:24 (4592): No heartbeat from core client for 30 sec - exiting 19:04:25 (4592): No heartbeat from core client for 30 sec - exiting 19:04:26 (4592): No heartbeat from core client for 30 sec - exiting 19:04:27 (4592): No heartbeat from core client for 30 sec - exiting 19:04:28 (4592): No heartbeat from core client for 30 sec - exiting 19:04:29 (4592): No heartbeat from core client for 30 sec - exiting 19:04:30 (4592): No heartbeat from core client for 30 sec - exiting 19:04:31 (4592): No heartbeat from core client for 30 sec - exiting 19:04:32 (4592): No heartbeat from core client for 30 sec - exiting 19:04:33 (4592): No heartbeat from core client for 30 sec - exiting 19:04:34 (4592): No heartbeat from core client for 30 sec - exiting 19:04:35 (4592): No heartbeat from core client for 30 sec - exiting 19:04:36 (4592): No heartbeat from core client for 30 sec - exiting 19:04:37 (4592): No heartbeat from core client for 30 sec - exiting 19:04:38 (4592): No heartbeat from core client for 30 sec - exiting 19:04:39 (4592): No heartbeat from core client for 30 sec - exiting 19:04:40 (4592): No heartbeat from core client for 30 sec - exiting 19:04:41 (4592): No heartbeat from core client for 30 sec - exiting 19:04:42 (4592): No heartbeat from core client for 30 sec - exiting 19:04:43 (4592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:04:44 (4592): No heartbeat from core client for 30 sec - exiting 19:04:45 (4592): No heartbeat from core client for 30 sec - exiting 19:04:46 (4592): No heartbeat from core client for 30 sec - exiting 19:04:47 (4592): No heartbeat from core client for 30 sec - exiting 19:04:48 (4592): No heartbeat from core client for 30 sec - exiting 19:04:49 (4592): No heartbeat from core client for 30 sec - exiting 19:04:50 (4592): No heartbeat from core client for 30 sec - exiting 19:04:51 (4592): No heartbeat from core client for 30 sec - exiting 19:04:52 (4592): No heartbeat from core client for 30 sec - exiting 19:04:54 (4592): No heartbeat from core client for 30 sec - exiting 19:11:53 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:29 (9952): No heartbeat from core client for 30 sec - exiting 19:13:30 (9952): No heartbeat from core client for 30 sec - exiting 19:13:31 (9952): No heartbeat from core client for 30 sec - exiting 19:13:32 (9952): No heartbeat from core client for 30 sec - exiting 19:13:33 (9952): No heartbeat from core client for 30 sec - exiting 19:13:34 (9952): No heartbeat from core client for 30 sec - exiting 19:13:35 (9952): No heartbeat from core client for 30 sec - exiting 19:13:36 (9952): No heartbeat from core client for 30 sec - exiting 19:13:37 (9952): No heartbeat from core client for 30 sec - exiting 19:13:38 (9952): No heartbeat from core client for 30 sec - exiting 19:13:39 (9952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:40 (9952): No heartbeat from core client for 30 sec - exiting 19:32:29 (15964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:33:20 (14424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:56:37 (8300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:57:33 (11060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:28 (5632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:22:38 (4612): No heartbeat from core client for 30 sec - exiting 13:22:39 (4612): No heartbeat from core client for 30 sec - exiting 13:22:40 (4612): No heartbeat from core client for 30 sec - exiting 13:22:41 (4612): No heartbeat from core client for 30 sec - exiting 13:22:42 (4612): No heartbeat from core client for 30 sec - exiting 13:22:43 (4612): No heartbeat from core client for 30 sec - exiting 13:22:44 (4612): No heartbeat from core client for 30 sec - exiting 13:22:45 (4612): No heartbeat from core client for 30 sec - exiting 13:22:46 (4612): No heartbeat from core client for 30 sec - exiting 13:22:47 (4612): No heartbeat from core client for 30 sec - exiting 13:22:48 (4612): No heartbeat from core client for 30 sec - exiting 13:22:49 (4612): No heartbeat from core client for 30 sec - exiting 13:22:50 (4612): No heartbeat from core client for 30 sec - exiting 13:22:51 (4612): No heartbeat from core client for 30 sec - exiting 13:22:52 (4612): No heartbeat from core client for 30 sec - exiting 13:22:53 (4612): No heartbeat from core client for 30 sec - exiting 13:22:54 (4612): No heartbeat from core client for 30 sec - exiting 13:22:55 (4612): No heartbeat from core client for 30 sec - exiting 13:22:56 (4612): No heartbeat from core client for 30 sec - exiting 13:22:57 (4612): No heartbeat from core client for 30 sec - exiting 13:22:58 (4612): No heartbeat from core client for 30 sec - exiting 13:22:59 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:00 (4612): No heartbeat from core client for 30 sec - exiting 13:23:01 (4612): No heartbeat from core client for 30 sec - exiting 13:23:02 (4612): No heartbeat from core client for 30 sec - exiting 13:46:31 (12064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:48:05 (12156): No heartbeat from core client for 30 sec - exiting 13:48:06 (12156): No heartbeat from core client for 30 sec - exiting 13:48:07 (12156): No heartbeat from core client for 30 sec - exiting 13:48:08 (12156): No heartbeat from core client for 30 sec - exiting 13:48:09 (12156): No heartbeat from core client for 30 sec - exiting 13:48:10 (12156): No heartbeat from core client for 30 sec - exiting 13:48:11 (12156): No heartbeat from core client for 30 sec - exiting 13:48:12 (12156): No heartbeat from core client for 30 sec - exiting 13:48:13 (12156): No heartbeat from core client for 30 sec - exiting 13:48:14 (12156): No heartbeat from core client for 30 sec - exiting 13:48:15 (12156): No heartbeat from core client for 30 sec - exiting 13:48:16 (12156): No heartbeat from core client for 30 sec - exiting 13:48:17 (12156): No heartbeat from core client for 30 sec - exiting 13:48:18 (12156): No heartbeat from core client for 30 sec - exiting 13:48:19 (12156): No heartbeat from core client for 30 sec - exiting 13:48:20 (12156): No heartbeat from core client for 30 sec - exiting 13:48:21 (12156): No heartbeat from core client for 30 sec - exiting 13:48:22 (12156): No heartbeat from core client for 30 sec - exiting 13:48:23 (12156): No heartbeat from core client for 30 sec - exiting 13:48:24 (12156): No heartbeat from core client for 30 sec - exiting 13:48:25 (12156): No heartbeat from core client for 30 sec - exiting 13:48:26 (12156): No heartbeat from core client for 30 sec - exiting 13:48:27 (12156): No heartbeat from core client for 30 sec - exiting 13:48:28 (12156): No heartbeat from core client for 30 sec - exiting 13:48:29 (12156): No heartbeat from core client for 30 sec - exiting 13:48:30 (12156): No heartbeat from core client for 30 sec - exiting 13:48:31 (12156): No heartbeat from core client for 30 sec - exiting 13:48:32 (12156): No heartbeat from core client for 30 sec - exiting 13:48:33 (12156): No heartbeat from core client for 30 sec - exiting 13:48:34 (12156): No heartbeat from core client for 30 sec - exiting 13:48:35 (12156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:48:36 (12156): No heartbeat from core client for 30 sec - exiting 13:48:37 (12156): No heartbeat from core client for 30 sec - exiting 13:48:38 (12156): No heartbeat from core client for 30 sec - exiting 13:48:39 (12156): No heartbeat from core client for 30 sec - exiting 13:48:40 (12156): No heartbeat from core client for 30 sec - exiting 13:48:41 (12156): No heartbeat from core client for 30 sec - exiting 13:48:42 (12156): No heartbeat from core client for 30 sec - exiting 13:48:43 (12156): No heartbeat from core client for 30 sec - exiting 13:48:44 (12156): No heartbeat from core client for 30 sec - exiting 13:48:45 (12156): No heartbeat from core client for 30 sec - exiting 13:48:46 (12156): No heartbeat from core client for 30 sec - exiting 13:50:47 (15096): No heartbeat from core client for 30 sec - exiting 13:50:48 (15096): No heartbeat from core client for 30 sec - exiting 13:50:49 (15096): No heartbeat from core client for 30 sec - exiting 13:50:50 (15096): No heartbeat from core client for 30 sec - exiting 13:50:51 (15096): No heartbeat from core client for 30 sec - exiting 13:50:52 (15096): No heartbeat from core client for 30 sec - exiting 13:50:53 (15096): No heartbeat from core client for 30 sec - exiting 13:50:54 (15096): No heartbeat from core client for 30 sec - exiting 13:50:55 (15096): No heartbeat from core client for 30 sec - exiting 13:50:56 (15096): No heartbeat from core client for 30 sec - exiting 13:50:57 (15096): No heartbeat from core client for 30 sec - exiting 13:50:58 (15096): No heartbeat from core client for 30 sec - exiting 13:50:59 (15096): No heartbeat from core client for 30 sec - exiting 13:51:00 (15096): No heartbeat from core client for 30 sec - exiting 13:51:01 (15096): No heartbeat from core client for 30 sec - exiting 13:51:02 (15096): No heartbeat from core client for 30 sec - exiting 13:51:03 (15096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:51:04 (15096): No heartbeat from core client for 30 sec - exiting 13:51:05 (15096): No heartbeat from core client for 30 sec - exiting 13:51:06 (15096): No heartbeat from core client for 30 sec - exiting 13:51:07 (15096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:47:50 (10476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:25:00 (12856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:17:06 (15644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:17:08 (15644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:30:36 (15368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:30:38 (15368): No heartbeat from core client for 30 sec - exiting 22:30:39 (15368): No heartbeat from core client for 30 sec - exiting 22:32:14 (13548): No heartbeat from core client for 30 sec - exiting 22:32:15 (13548): No heartbeat from core client for 30 sec - exiting 22:32:16 (13548): No heartbeat from core client for 30 sec - exiting 22:32:17 (13548): No heartbeat from core client for 30 sec - exiting 22:32:18 (13548): No heartbeat from core client for 30 sec - exiting 22:32:19 (13548): No heartbeat from core client for 30 sec - exiting 22:32:20 (13548): No heartbeat from core client for 30 sec - exiting 22:32:21 (13548): No heartbeat from core client for 30 sec - exiting 22:32:22 (13548): No heartbeat from core client for 30 sec - exiting 22:32:23 (13548): No heartbeat from core client for 30 sec - exiting 22:32:24 (13548): No heartbeat from core client for 30 sec - exiting 22:32:25 (13548): No heartbeat from core client for 30 sec - exiting 22:32:26 (13548): No heartbeat from core client for 30 sec - exiting 22:32:27 (13548): No heartbeat from core client for 30 sec - exiting 22:32:28 (13548): No heartbeat from core client for 30 sec - exiting 22:32:30 (13548): No heartbeat from core client for 30 sec - exiting 22:32:31 (13548): No heartbeat from core client for 30 sec - exiting 22:32:32 (13548): No heartbeat from core client for 30 sec - exiting 22:32:33 (13548): No heartbeat from core client for 30 sec - exiting 22:32:34 (13548): No heartbeat from core client for 30 sec - exiting 22:32:35 (13548): No heartbeat from core client for 30 sec - exiting 22:32:36 (13548): No heartbeat from core client for 30 sec - exiting 22:32:37 (13548): No heartbeat from core client for 30 sec - exiting 22:32:38 (13548): No heartbeat from core client for 30 sec - exiting 22:32:39 (13548): No heartbeat from core client for 30 sec - exiting 22:32:40 (13548): No heartbeat from core client for 30 sec - exiting 22:32:41 (13548): No heartbeat from core client for 30 sec - exiting 22:32:42 (13548): No heartbeat from core client for 30 sec - exiting 22:32:43 (13548): No heartbeat from core client for 30 sec - exiting 22:32:44 (13548): No heartbeat from core client for 30 sec - exiting 22:32:45 (13548): No heartbeat from core client for 30 sec - exiting 22:32:46 (13548): No heartbeat from core client for 30 sec - exiting 22:32:47 (13548): No heartbeat from core client for 30 sec - exiting 22:32:48 (13548): No heartbeat from core client for 30 sec - exiting 22:32:49 (13548): No heartbeat from core client for 30 sec - exiting 22:32:50 (13548): No heartbeat from core client for 30 sec - exiting 22:32:51 (13548): No heartbeat from core client for 30 sec - exiting 22:32:52 (13548): No heartbeat from core client for 30 sec - exiting 22:32:53 (13548): No heartbeat from core client for 30 sec - exiting 22:32:54 (13548): No heartbeat from core client for 30 sec - exiting 22:32:55 (13548): No heartbeat from core client for 30 sec - exiting 22:32:56 (13548): No heartbeat from core client for 30 sec - exiting 22:32:57 (13548): No heartbeat from core client for 30 sec - exiting 22:32:58 (13548): No heartbeat from core client for 30 sec - exiting 22:32:59 (13548): No heartbeat from core client for 30 sec - exiting 22:33:00 (13548): No heartbeat from core client for 30 sec - exiting 22:33:01 (13548): No heartbeat from core client for 30 sec - exiting 22:33:02 (13548): No heartbeat from core client for 30 sec - exiting 22:33:03 (13548): No heartbeat from core client for 30 sec - exiting 22:33:04 (13548): No heartbeat from core client for 30 sec - exiting 22:33:05 (13548): No heartbeat from core client for 30 sec - exiting 22:33:06 (13548): No heartbeat from core client for 30 sec - exiting 22:33:07 (13548): No heartbeat from core client for 30 sec - exiting 22:33:08 (13548): No heartbeat from core client for 30 sec - exiting 22:33:09 (13548): No heartbeat from core client for 30 sec - exiting 22:33:10 (13548): No heartbeat from core client for 30 sec - exiting 22:33:11 (13548): No heartbeat from core client for 30 sec - exiting 22:33:12 (13548): No heartbeat from core client for 30 sec - exiting 22:33:13 (13548): No heartbeat from core client for 30 sec - exiting 22:33:14 (13548): No heartbeat from core client for 30 sec - exiting 22:33:15 (13548): No heartbeat from core client for 30 sec - exiting 22:33:16 (13548): No heartbeat from core client for 30 sec - exiting 22:33:17 (13548): No heartbeat from core client for 30 sec - exiting 22:33:18 (13548): No heartbeat from core client for 30 sec - exiting 22:33:19 (13548): No heartbeat from core client for 30 sec - exiting 22:33:20 (13548): No heartbeat from core client for 30 sec - exiting 22:33:21 (13548): No heartbeat from core client for 30 sec - exiting 22:33:22 (13548): No heartbeat from core client for 30 sec - exiting 22:33:23 (13548): No heartbeat from core client for 30 sec - exiting 22:33:24 (13548): No heartbeat from core client for 30 sec - exiting 22:33:25 (13548): No heartbeat from core client for 30 sec - exiting 22:33:26 (13548): No heartbeat from core client for 30 sec - exiting 22:33:27 (13548): No heartbeat from core client for 30 sec - exiting 22:33:28 (13548): No heartbeat from core client for 30 sec - exiting 22:33:29 (13548): No heartbeat from core client for 30 sec - exiting 22:33:30 (13548): No heartbeat from core client for 30 sec - exiting 22:33:31 (13548): No heartbeat from core client for 30 sec - exiting 22:33:32 (13548): No heartbeat from core client for 30 sec - exiting 22:33:33 (13548): No heartbeat from core client for 30 sec - exiting 22:33:34 (13548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:54:51 (6208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:10:58 (13084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:00 (13084): No heartbeat from core client for 30 sec - exiting 15:11:01 (13084): No heartbeat from core client for 30 sec - exiting 16:13:03 (16204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:13:04 (16204): No heartbeat from core client for 30 sec - exiting 16:15:54 (2948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:17:12 (15412): No heartbeat from core client for 30 sec - exiting 16:17:13 (15412): No heartbeat from core client for 30 sec - exiting 16:17:14 (15412): No heartbeat from core client for 30 sec - exiting 16:17:15 (15412): No heartbeat from core client for 30 sec - exiting 16:17:16 (15412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:35:34 (2944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:15:16 (15472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Nov 2011 16:53:22	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	907,200	1,269,626	1.3995
29 Nov 2011 01:13:34	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	881,280	1,233,780	1.4000
28 Nov 2011 10:26:06	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	855,360	1,197,998	1.4006
27 Nov 2011 22:22:14	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	829,440	1,162,609	1.4017
27 Nov 2011 05:49:47	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	803,520	1,126,629	1.4021
26 Nov 2011 12:40:33	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	777,600	1,090,698	1.4026
25 Nov 2011 20:20:48	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	751,680	1,054,659	1.4031
25 Nov 2011 04:19:48	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	725,760	1,018,953	1.4040
24 Nov 2011 11:22:19	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	699,840	982,770	1.4043
23 Nov 2011 16:34:09	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	673,920	946,412	1.4043
23 Nov 2011 00:30:32	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	648,000	910,522	1.4051
22 Nov 2011 08:33:46	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	622,080	874,249	1.4054
21 Nov 2011 14:09:52	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	596,160	837,655	1.4051
20 Nov 2011 20:35:07	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	570,240	801,782	1.4060
20 Nov 2011 03:16:38	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	544,320	765,219	1.4058
19 Nov 2011 10:44:05	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	518,400	729,417	1.4071
18 Nov 2011 18:01:48	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	492,480	692,918	1.4070
18 Nov 2011 02:32:32	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	466,560	656,690	1.4075
17 Nov 2011 11:09:00	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	440,640	621,016	1.4094
16 Nov 2011 17:33:22	689984	13569578	hadcm3n_yj86_1900_40_007514944_4	414,720	584,735	1.4100