Task 13666972

Name	hadcm3n_y8q1_1900_40_007525620_4
Workunit	7723095
Created	28 Nov 2011, 22:43:24 UTC
Sent	28 Nov 2011, 22:43:31 UTC
Report deadline	28 Feb 2012, 6:10:42 UTC
Received	9 Jan 2012, 20:18:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1167875
Run time	6 days 6 hours 26 min 55 sec
CPU time	5 days 17 hours 6 min 37 sec
Validate state	Invalid
Credit	5,909.76
Device peak FLOPS	4.16 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> =0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:31:48 (436): No heartbeat from core client for 30 sec - exiting 13:31:49 (436): No heartbeat from core client for 30 sec - exiting 13:31:50 (436): No heartbeat from core client for 30 sec - exiting 13:31:51 (436): No heartbeat from core client for 30 sec - exiting 13:31:52 (436): No heartbeat from core client for 30 sec - exiting 13:31:53 (436): No heartbeat from core client for 30 sec - exiting 13:31:54 (436): No heartbeat from core client for 30 sec - exiting 13:31:55 (436): No heartbeat from core client for 30 sec - exiting 13:31:56 (436): No heartbeat from core client for 30 sec - exiting 13:31:57 (436): No heartbeat from core client for 30 sec - exiting 13:31:58 (436): No heartbeat from core client for 30 sec - exiting 13:31:59 (436): No heartbeat from core client for 30 sec - exiting 13:32:00 (436): No heartbeat from core client for 30 sec - exiting 13:32:01 (436): No heartbeat from core client for 30 sec - exiting 13:32:02 (436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:32:03 (436): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 17:51:18 (2896): No heartbeat from core client for 30 sec - exiting 17:51:19 (2896): No heartbeat from core client for 30 sec - exiting 17:51:20 (2896): No heartbeat from core client for 30 sec - exiting 17:51:21 (2896): No heartbeat from core client for 30 sec - exiting 17:51:22 (2896): No heartbeat from core client for 30 sec - exiting 17:51:23 (2896): No heartbeat from core client for 30 sec - exiting 17:51:24 (2896): No heartbeat from core client for 30 sec - exiting 17:51:25 (2896): No heartbeat from core client for 30 sec - exiting 17:51:26 (2896): No heartbeat from core client for 30 sec - exiting 17:51:27 (2896): No heartbeat from core client for 30 sec - exiting 17:51:28 (2896): No heartbeat from core client for 30 sec - exiting 17:51:29 (2896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:24:03 (2992): No heartbeat from core client for 30 sec - exiting 19:24:04 (2992): No heartbeat from core client for 30 sec - exiting 19:24:05 (2992): No heartbeat from core client for 30 sec - exiting 19:24:06 (2992): No heartbeat from core client for 30 sec - exiting 19:24:07 (2992): No heartbeat from core client for 30 sec - exiting 19:24:08 (2992): No heartbeat from core client for 30 sec - exiting 19:24:09 (2992): No heartbeat from core client for 30 sec - exiting 19:24:10 (2992): No heartbeat from core client for 30 sec - exiting 19:24:11 (2992): No heartbeat from core client for 30 sec - exiting 19:24:12 (2992): No heartbeat from core client for 30 sec - exiting 19:24:13 (2992): No heartbeat from core client for 30 sec - exiting 19:24:14 (2992): No heartbeat from core client for 30 sec - exiting 19:24:15 (2992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:16 (2992): No heartbeat from core client for 30 sec - exiting 20:00:47 (5868): No heartbeat from core client for 30 sec - exiting 20:00:48 (5868): No heartbeat from core client for 30 sec - exiting 20:00:49 (5868): No heartbeat from core client for 30 sec - exiting 20:00:50 (5868): No heartbeat from core client for 30 sec - exiting 20:00:51 (5868): No heartbeat from core client for 30 sec - exiting 20:00:52 (5868): No heartbeat from core client for 30 sec - exiting 20:00:53 (5868): No heartbeat from core client for 30 sec - exiting 20:00:54 (5868): No heartbeat from core client for 30 sec - exiting 20:00:55 (5868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:38:34 (5780): No heartbeat from core client for 30 sec - exiting 21:38:35 (5780): No heartbeat from core client for 30 sec - exiting 21:38:36 (5780): No heartbeat from core client for 30 sec - exiting 21:38:37 (5780): No heartbeat from core client for 30 sec - exiting 21:38:38 (5780): No heartbeat from core client for 30 sec - exiting 21:38:39 (5780): No heartbeat from core client for 30 sec - exiting 21:38:40 (5780): No heartbeat from core client for 30 sec - exiting 21:38:41 (5780): No heartbeat from core client for 30 sec - exiting 21:38:42 (5780): No heartbeat from core client for 30 sec - exiting 21:38:43 (5780): No heartbeat from core client for 30 sec - exiting 21:38:44 (5780): No heartbeat from core client for 30 sec - exiting 21:38:45 (5780): No heartbeat from core client for 30 sec - exiting 21:38:46 (5780): No heartbeat from core client for 30 sec - exiting 21:38:47 (5780): No heartbeat from core client for 30 sec - exiting 21:38:48 (5780): No heartbeat from core client for 30 sec - exiting 21:38:49 (5780): No heartbeat from core client for 30 sec - exiting 21:38:50 (5780): No heartbeat from core client for 30 sec - exiting 21:38:51 (5780): No heartbeat from core client for 30 sec - exiting 21:38:52 (5780): No heartbeat from core client for 30 sec - exiting 21:38:53 (5780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:38:54 (5780): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 10:09:20 (5824): No heartbeat from core client for 30 sec - exiting 10:09:21 (5824): No heartbeat from core client for 30 sec - exiting 10:09:22 (5824): No heartbeat from core client for 30 sec - exiting 10:09:23 (5824): No heartbeat from core client for 30 sec - exiting 10:09:24 (5824): No heartbeat from core client for 30 sec - exiting 10:09:25 (5824): No heartbeat from core client for 30 sec - exiting 10:09:26 (5824): No heartbeat from core client for 30 sec - exiting 10:09:27 (5824): No heartbeat from core client for 30 sec - exiting 10:09:28 (5824): No heartbeat from core client for 30 sec - exiting 10:09:29 (5824): No heartbeat from core client for 30 sec - exiting 10:09:30 (5824): No heartbeat from core client for 30 sec - exiting 10:09:31 (5824): No heartbeat from core client for 30 sec - exiting 10:09:32 (5824): No heartbeat from core client for 30 sec - exiting 10:09:33 (5824): No heartbeat from core client for 30 sec - exiting 10:09:34 (5824): No heartbeat from core client for 30 sec - exiting 10:09:35 (5824): No heartbeat from core client for 30 sec - exiting 10:09:36 (5824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:31:50 (4904): No heartbeat from core client for 30 sec - exiting 15:31:51 (4904): No heartbeat from core client for 30 sec - exiting 15:31:52 (4904): No heartbeat from core client for 30 sec - exiting 15:31:53 (4904): No heartbeat from core client for 30 sec - exiting 15:31:54 (4904): No heartbeat from core client for 30 sec - exiting 15:31:55 (4904): No heartbeat from core client for 30 sec - exiting 15:31:56 (4904): No heartbeat from core client for 30 sec - exiting 15:31:57 (4904): No heartbeat from core client for 30 sec - exiting 15:31:58 (4904): No heartbeat from core client for 30 sec - exiting 15:31:59 (4904): No heartbeat from core client for 30 sec - exiting 15:32:00 (4904): No heartbeat from core client for 30 sec - exiting 15:32:01 (4904): No heartbeat from core client for 30 sec - exiting 15:32:02 (4904): No heartbeat from core client for 30 sec - exiting 15:32:03 (4904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:21:55 (5984): No heartbeat from core client for 30 sec - exiting 20:21:56 (5984): No heartbeat from core client for 30 sec - exiting 20:21:57 (5984): No heartbeat from core client for 30 sec - exiting 20:21:58 (5984): No heartbeat from core client for 30 sec - exiting 20:21:59 (5984): No heartbeat from core client for 30 sec - exiting 20:22:00 (5984): No heartbeat from core client for 30 sec - exiting 20:22:01 (5984): No heartbeat from core client for 30 sec - exiting 20:22:02 (5984): No heartbeat from core client for 30 sec - exiting 20:22:03 (5984): No heartbeat from core client for 30 sec - exiting 20:22:04 (5984): No heartbeat from core client for 30 sec - exiting 20:22:05 (5984): No heartbeat from core client for 30 sec - exiting 20:22:06 (5984): No heartbeat from core client for 30 sec - exiting 20:22:07 (5984): No heartbeat from core client for 30 sec - exiting 20:22:08 (5984): No heartbeat from core client for 30 sec - exiting 20:22:09 (5984): No heartbeat from core client for 30 sec - exiting 20:22:10 (5984): No heartbeat from core client for 30 sec - exiting 20:22:11 (5984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:51:18 (5728): No heartbeat from core client for 30 sec - exiting 10:51:19 (5728): No heartbeat from core client for 30 sec - exiting 10:51:20 (5728): No heartbeat from core client for 30 sec - exiting 10:51:21 (5728): No heartbeat from core client for 30 sec - exiting 10:51:22 (5728): No heartbeat from core client for 30 sec - exiting 10:51:23 (5728): No heartbeat from core client for 30 sec - exiting 10:51:24 (5728): No heartbeat from core client for 30 sec - exiting 10:51:25 (5728): No heartbeat from core client for 30 sec - exiting 10:51:26 (5728): No heartbeat from core client for 30 sec - exiting 10:51:27 (5728): No heartbeat from core client for 30 sec - exiting 10:51:28 (5728): No heartbeat from core client for 30 sec - exiting 10:51:29 (5728): No heartbeat from core client for 30 sec - exiting 10:51:30 (5728): No heartbeat from core client for 30 sec - exiting 10:51:31 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:37:55 (5812): No heartbeat from core client for 30 sec - exiting 14:37:56 (5812): No heartbeat from core client for 30 sec - exiting 14:37:57 (5812): No heartbeat from core client for 30 sec - exiting 14:37:58 (5812): No heartbeat from core client for 30 sec - exiting 14:37:59 (5812): No heartbeat from core client for 30 sec - exiting 14:38:00 (5812): No heartbeat from core client for 30 sec - exiting 14:38:01 (5812): No heartbeat from core client for 30 sec - exiting 14:38:02 (5812): No heartbeat from core client for 30 sec - exiting 14:38:03 (5812): No heartbeat from core client for 30 sec - exiting 14:38:04 (5812): No heartbeat from core client for 30 sec - exiting 14:38:05 (5812): No heartbeat from core client for 30 sec - exiting 14:38:06 (5812): No heartbeat from core client for 30 sec - exiting 14:38:07 (5812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:39:49 (5544): No heartbeat from core client for 30 sec - exiting 18:39:50 (5544): No heartbeat from core client for 30 sec - exiting 18:39:51 (5544): No heartbeat from core client for 30 sec - exiting 18:39:52 (5544): No heartbeat from core client for 30 sec - exiting 18:39:53 (5544): No heartbeat from core client for 30 sec - exiting 18:39:54 (5544): No heartbeat from core client for 30 sec - exiting 18:39:55 (5544): No heartbeat from core client for 30 sec - exiting 18:39:56 (5544): No heartbeat from core client for 30 sec - exiting 18:39:57 (5544): No heartbeat from core client for 30 sec - exiting 18:39:58 (5544): No heartbeat from core client for 30 sec - exiting 18:39:59 (5544): No heartbeat from core client for 30 sec - exiting 18:40:00 (5544): No heartbeat from core client for 30 sec - exiting 18:40:01 (5544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:00:50 (5856): No heartbeat from core client for 30 sec - exiting 09:00:51 (5856): No heartbeat from core client for 30 sec - exiting 09:00:52 (5856): No heartbeat from core client for 30 sec - exiting 09:00:53 (5856): No heartbeat from core client for 30 sec - exiting 09:00:54 (5856): No heartbeat from core client for 30 sec - exiting 09:00:55 (5856): No heartbeat from core client for 30 sec - exiting 09:00:56 (5856): No heartbeat from core client for 30 sec - exiting 09:00:57 (5856): No heartbeat from core client for 30 sec - exiting 09:00:58 (5856): No heartbeat from core client for 30 sec - exiting 09:00:59 (5856): No heartbeat from core client for 30 sec - exiting 09:01:00 (5856): No heartbeat from core client for 30 sec - exiting 09:01:01 (5856): No heartbeat from core client for 30 sec - exiting 09:01:02 (5856): No heartbeat from core client for 30 sec - exiting 09:01:03 (5856): No heartbeat from core client for 30 sec - exiting 09:01:04 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:50:32 (4912): No heartbeat from core client for 30 sec - exiting 11:50:34 (4912): No heartbeat from core client for 30 sec - exiting 11:50:35 (4912): No heartbeat from core client for 30 sec - exiting 11:50:36 (4912): No heartbeat from core client for 30 sec - exiting 11:50:37 (4912): No heartbeat from core client for 30 sec - exiting 11:50:38 (4912): No heartbeat from core client for 30 sec - exiting 11:50:39 (4912): No heartbeat from core client for 30 sec - exiting 11:50:40 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:19:33 (5744): No heartbeat from core client for 30 sec - exiting 14:19:34 (5744): No heartbeat from core client for 30 sec - exiting 14:19:35 (5744): No heartbeat from core client for 30 sec - exiting 14:19:36 (5744): No heartbeat from core client for 30 sec - exiting 14:19:37 (5744): No heartbeat from core client for 30 sec - exiting 14:19:38 (5744): No heartbeat from core client for 30 sec - exiting 14:19:39 (5744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:09:54 (5764): No heartbeat from core client for 30 sec - exiting 18:09:55 (5764): No heartbeat from core client for 30 sec - exiting 18:09:56 (5764): No heartbeat from core client for 30 sec - exiting 18:09:57 (5764): No heartbeat from core client for 30 sec - exiting 18:09:58 (5764): No heartbeat from core client for 30 sec - exiting 18:09:59 (5764): No heartbeat from core client for 30 sec - exiting 18:10:00 (5764): No heartbeat from core client for 30 sec - exiting 18:10:01 (5764): No heartbeat from core client for 30 sec - exiting 18:47:05 (6140): No heartbeat from core client for 30 sec - exiting 18:47:06 (6140): No heartbeat from core client for 30 sec - exiting 18:47:07 (6140): No heartbeat from core client for 30 sec - exiting 18:47:08 (6140): No heartbeat from core client for 30 sec - exiting 18:47:09 (6140): No heartbeat from core client for 30 sec - exiting 18:47:10 (6140): No heartbeat from core client for 30 sec - exiting 18:47:11 (6140): No heartbeat from core client for 30 sec - exiting 18:47:12 (6140): No heartbeat from core client for 30 sec - exiting 18:47:13 (6140): No heartbeat from core client for 30 sec - exiting 18:47:14 (6140): No heartbeat from core client for 30 sec - exiting 18:47:15 (6140): No heartbeat from core client for 30 sec - exiting 18:47:16 (6140): No heartbeat from core client for 30 sec - exiting 18:47:17 (6140): No heartbeat from core client for 30 sec - exiting 18:47:18 (6140): No heartbeat from core client for 30 sec - exiting 18:47:19 (6140): No heartbeat from core client for 30 sec - exiting 18:47:20 (6140): No heartbeat from core client for 30 sec - exiting 18:47:21 (6140): No heartbeat from core client for 30 sec - exiting 18:47:22 (6140): No heartbeat from core client for 30 sec - exiting 18:47:23 (6140): No heartbeat from core client for 30 sec - exiting 18:47:24 (6140): No heartbeat from core client for 30 sec - exiting 18:47:25 (6140): No heartbeat from core client for 30 sec - exiting 18:47:26 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:48:53 (5076): No heartbeat from core client for 30 sec - exiting 09:48:54 (5076): No heartbeat from core client for 30 sec - exiting 09:48:55 (5076): No heartbeat from core client for 30 sec - exiting 09:48:56 (5076): No heartbeat from core client for 30 sec - exiting 09:48:57 (5076): No heartbeat from core client for 30 sec - exiting 09:48:58 (5076): No heartbeat from core client for 30 sec - exiting 09:48:59 (5076): No heartbeat from core client for 30 sec - exiting 09:49:00 (5076): No heartbeat from core client for 30 sec - exiting 09:49:01 (5076): No heartbeat from core client for 30 sec - exiting 09:49:02 (5076): No heartbeat from core client for 30 sec - exiting 09:49:03 (5076): No heartbeat from core client for 30 sec - exiting 09:49:04 (5076): No heartbeat from core client for 30 sec - exiting 09:49:05 (5076): No heartbeat from core client for 30 sec - exiting 09:49:06 (5076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:45:25 (6088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:50:05 (5820): No heartbeat from core client for 30 sec - exiting 17:50:06 (5820): No heartbeat from core client for 30 sec - exiting 17:50:07 (5820): No heartbeat from core client for 30 sec - exiting 17:50:08 (5820): No heartbeat from core client for 30 sec - exiting 17:50:09 (5820): No heartbeat from core client for 30 sec - exiting 17:50:10 (5820): No heartbeat from core client for 30 sec - exiting 17:50:11 (5820): No heartbeat from core client for 30 sec - exiting 17:50:12 (5820): No heartbeat from core client for 30 sec - exiting 17:50:13 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:11:53 (6008): No heartbeat from core client for 30 sec - exiting 09:11:55 (6008): No heartbeat from core client for 30 sec - exiting 09:11:56 (6008): No heartbeat from core client for 30 sec - exiting 09:11:57 (6008): No heartbeat from core client for 30 sec - exiting 09:11:58 (6008): No heartbeat from core client for 30 sec - exiting 09:11:59 (6008): No heartbeat from core client for 30 sec - exiting 09:12:00 (6008): No heartbeat from core client for 30 sec - exiting 09:12:01 (6008): No heartbeat from core client for 30 sec - exiting 09:12:02 (6008): No heartbeat from core client for 30 sec - exiting 09:12:03 (6008): No heartbeat from core client for 30 sec - exiting 09:12:04 (6008): No heartbeat from core client for 30 sec - exiting 09:12:05 (6008): No heartbeat from core client for 30 sec - exiting 09:12:06 (6008): No heartbeat from core client for 30 sec - exiting 09:12:07 (6008): No heartbeat from core client for 30 sec - exiting 09:12:08 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:17:44 (4868): No heartbeat from core client for 30 sec - exiting 10:17:45 (4868): No heartbeat from core client for 30 sec - exiting 10:17:46 (4868): No heartbeat from core client for 30 sec - exiting 10:17:47 (4868): No heartbeat from core client for 30 sec - exiting 10:17:48 (4868): No heartbeat from core client for 30 sec - exiting 10:17:49 (4868): No heartbeat from core client for 30 sec - exiting 10:17:50 (4868): No heartbeat from core client for 30 sec - exiting 10:17:51 (4868): No heartbeat from core client for 30 sec - exiting 10:17:52 (4868): No heartbeat from core client for 30 sec - exiting 10:17:53 (4868): No heartbeat from core client for 30 sec - exiting 10:17:54 (4868): No heartbeat from core client for 30 sec - exiting 10:17:55 (4868): No heartbeat from core client for 30 sec - exiting 10:17:56 (4868): No heartbeat from core client for 30 sec - exiting 10:17:57 (4868): No heartbeat from core client for 30 sec - exiting 10:17:58 (4868): No heartbeat from core client for 30 sec - exiting 10:17:59 (4868): No heartbeat from core client for 30 sec - exiting 10:18:00 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:15:38 (4308): No heartbeat from core client for 30 sec - exiting 19:15:40 (4308): No heartbeat from core client for 30 sec - exiting 19:15:41 (4308): No heartbeat from core client for 30 sec - exiting 19:15:42 (4308): No heartbeat from core client for 30 sec - exiting 19:15:43 (4308): No heartbeat from core client for 30 sec - exiting 19:15:44 (4308): No heartbeat from core client for 30 sec - exiting 19:15:45 (4308): No heartbeat from core client for 30 sec - exiting 19:15:46 (4308): No heartbeat from core client for 30 sec - exiting 19:15:47 (4308): No heartbeat from core client for 30 sec - exiting 19:15:48 (4308): No heartbeat from core client for 30 sec - exiting 19:15:49 (4308): No heartbeat from core client for 30 sec - exiting 19:15:50 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:15:51 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:35:06 (1504): No heartbeat from core client for 30 sec - exiting 20:35:07 (1504): No heartbeat from core client for 30 sec - exiting 20:35:08 (1504): No heartbeat from core client for 30 sec - exiting 20:35:09 (1504): No heartbeat from core client for 30 sec - exiting 20:35:10 (1504): No heartbeat from core client for 30 sec - exiting 20:35:11 (1504): No heartbeat from core client for 30 sec - exiting 20:35:12 (1504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:32:07 (5560): No heartbeat from core client for 30 sec - exiting 19:32:08 (5560): No heartbeat from core client for 30 sec - exiting 19:32:09 (5560): No heartbeat from core client for 30 sec - exiting 19:32:10 (5560): No heartbeat from core client for 30 sec - exiting 19:32:11 (5560): No heartbeat from core client for 30 sec - exiting 19:32:12 (5560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:13 (5560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:23:31 (4152): No heartbeat from core client for 30 sec - exiting 19:23:32 (4152): No heartbeat from core client for 30 sec - exiting 19:23:33 (4152): No heartbeat from core client for 30 sec - exiting 19:23:34 (4152): No heartbeat from core client for 30 sec - exiting 19:23:35 (4152): No heartbeat from core client for 30 sec - exiting 19:23:36 (4152): No heartbeat from core client for 30 sec - exiting 19:23:37 (4152): No heartbeat from core client for 30 sec - exiting 19:23:38 (4152): No heartbeat from core client for 30 sec - exiting 19:23:39 (4152): No heartbeat from core client for 30 sec - exiting 19:23:40 (4152): No heartbeat from core client for 30 sec - exiting 19:23:41 (4152): No heartbeat from core client for 30 sec - exiting 19:23:42 (4152): No heartbeat from core client for 30 sec - exiting 19:23:43 (4152): No heartbeat from core client for 30 sec - exiting 19:23:44 (4152): No heartbeat from core client for 30 sec - exiting 19:23:45 (4152): No heartbeat from core client for 30 sec - exiting 19:23:46 (4152): No heartbeat from core client for 30 sec - exiting 19:23:47 (4152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4800, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:07:47 (5008): No heartbeat from core client for 30 sec - exiting 22:07:49 (5008): No heartbeat from core client for 30 sec - exiting 22:07:50 (5008): No heartbeat from core client for 30 sec - exiting 22:07:51 (5008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:52 (5008): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 07:02:23 (5312): No heartbeat from core client for 30 sec - exiting 07:02:24 (5312): No heartbeat from core client for 30 sec - exiting 07:02:25 (5312): No heartbeat from core client for 30 sec - exiting 07:02:26 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:02:27 (5312): No heartbeat from core client for 30 sec - exiting 19:15:18 (5764): No heartbeat from core client for 30 sec - exiting 19:15:20 (5764): No heartbeat from core client for 30 sec - exiting 19:15:21 (5764): No heartbeat from core client for 30 sec - exiting 19:15:22 (5764): No heartbeat from core client for 30 sec - exiting 19:15:23 (5764): No heartbeat from core client for 30 sec - exiting 19:15:24 (5764): No heartbeat from core client for 30 sec - exiting 19:15:25 (5764): No heartbeat from core client for 30 sec - exiting 19:15:26 (5764): No heartbeat from core client for 30 sec - exiting 19:15:27 (5764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:15:28 (5764): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:37:23 (5816): No heartbeat from core client for 30 sec - exiting 19:37:24 (5816): No heartbeat from core client for 30 sec - exiting 19:37:25 (5816): No heartbeat from core client for 30 sec - exiting 19:37:26 (5816): No heartbeat from core client for 30 sec - exiting 19:37:27 (5816): No heartbeat from core client for 30 sec - exiting 19:37:28 (5816): No heartbeat from core client for 30 sec - exiting 19:37:29 (5816): No heartbeat from core client for 30 sec - exiting 19:37:30 (5816): No heartbeat from core client for 30 sec - exiting 19:37:31 (5816): No heartbeat from core client for 30 sec - exiting 19:37:32 (5816): No heartbeat from core client for 30 sec - exiting 19:37:33 (5816): No heartbeat from core client for 30 sec - exiting 19:37:34 (5816): No heartbeat from core client for 30 sec - exiting 19:37:35 (5816): No heartbeat from core client for 30 sec - exiting 19:37:36 (5816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6012, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6012, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6012, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6012, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6012, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6012, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Jan 2012 21:29:52	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	492,480	475,681	0.9659
02 Jan 2012 17:42:12	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	466,560	450,158	0.9648
01 Jan 2012 19:50:35	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	440,640	424,875	0.9642
31 Dec 2011 19:06:46	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	414,720	399,865	0.9642
30 Dec 2011 23:03:39	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	388,800	374,754	0.9639
30 Dec 2011 15:43:05	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	362,880	349,614	0.9634
29 Dec 2011 14:05:31	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	336,960	324,588	0.9633
28 Dec 2011 12:05:33	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	311,040	300,018	0.9646
26 Dec 2011 22:54:50	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	285,120	275,254	0.9654
25 Dec 2011 20:22:21	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	259,200	250,249	0.9655
25 Dec 2011 09:29:07	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	233,280	225,313	0.9658
19 Dec 2011 10:28:03	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	207,360	199,716	0.9631
16 Dec 2011 23:11:16	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	181,440	174,920	0.9641
12 Dec 2011 22:06:50	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	155,520	149,893	0.9638
10 Dec 2011 20:58:27	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	129,600	125,342	0.9671
06 Dec 2011 21:41:33	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	103,680	100,168	0.9661
04 Dec 2011 12:50:54	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	77,760	75,520	0.9712
03 Dec 2011 07:20:57	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	51,840	50,070	0.9659
02 Dec 2011 23:53:59	1167875	13666972	hadcm3n_y8q1_1900_40_007525620_4	25,920	24,616	0.9497