Task 14935737

Name	hadcm3n_203f_1940_40_007957621_4
Workunit	8112733
Created	18 Jul 2012, 10:04:01 UTC
Sent	18 Jul 2012, 10:04:40 UTC
Report deadline	17 Oct 2012, 17:31:51 UTC
Received	10 Sep 2012, 20:40:55 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	941153
Run time	26 days 4 hours 39 min 38 sec
CPU time	23 days 1 hours 59 min 6 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.17 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <stderr_txt> 0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... hadcm3n_6.07_i686-apple-darwin(93527,0xa0852540) malloc: * error for object 0x183b404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(93527,0xa0852540) malloc: * error for object 0x183b400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(93527,0xa0852540) malloc: * error for object 0x183b404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(93527,0xa0852540) malloc: * error for object 0x183b400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(93527,0xa0852540) malloc: * error for object 0x861004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation 07:41:17 (93527): No heartbeat from core client for 30 sec - exiting 07:41:19 (93527): No heartbeat from core client for 30 sec - exiting 07:41:20 (93527): No heartbeat from core client for 30 sec - exiting 07:41:21 (93527): No heartbeat from core client for 30 sec - exiting 07:41:22 (93527): No heartbeat from core client for 30 sec - exiting 07:41:23 (93527): No heartbeat from core client for 30 sec - exiting 07:41:24 (93527): No heartbeat from core client for 30 sec - exiting 07:41:25 (93527): No heartbeat from core client for 30 sec - exiting 07:41:26 (93527): No heartbeat from core client for 30 sec - exiting 07:41:27 (93527): No heartbeat from core client for 30 sec - exiting 07:41:28 (93527): No heartbeat from core client for 30 sec - exiting 07:41:29 (93527): No heartbeat from core client for 30 sec - exiting 07:41:30 (93527): No heartbeat from core client for 30 sec - exiting 07:41:31 (93527): No heartbeat from core client for 30 sec - exiting 07:41:32 (93527): No heartbeat from core client for 30 sec - exiting 07:41:33 (93527): No heartbeat from core client for 30 sec - exiting 07:41:34 (93527): No heartbeat from core client for 30 sec - exiting 07:41:35 (93527): No heartbeat from core client for 30 sec - exiting 07:41:36 (93527): No heartbeat from core client for 30 sec - exiting 07:41:37 (93527): No heartbeat from core client for 30 sec - exiting 07:41:38 (93527): No heartbeat from core client for 30 sec - exiting 07:41:39 (93527): No heartbeat from core client for 30 sec - exiting 07:41:40 (93527): No heartbeat from core client for 30 sec - exiting 07:41:41 (93527): No heartbeat from core client for 30 sec - exiting 07:41:42 (93527): No heartbeat from core client for 30 sec - exiting 07:41:43 (93527): No heartbeat from core client for 30 sec - exiting 07:41:44 (93527): No heartbeat from core client for 30 sec - exiting 07:41:45 (93527): No heartbeat from core client for 30 sec - exiting 07:41:46 (93527): No heartbeat from core client for 30 sec - exiting 07:41:47 (93527): No heartbeat from core client for 30 sec - exiting 07:41:48 (93527): No heartbeat from core client for 30 sec - exiting 11:44:25 (93527): No heartbeat from core client for 30 sec - exiting 11:44:26 (93527): No heartbeat from core client for 30 sec - exiting 11:44:27 (93527): No heartbeat from core client for 30 sec - exiting 11:44:28 (93527): No heartbeat from core client for 30 sec - exiting 15:15:49 (93527): No heartbeat from core client for 30 sec - exiting 15:15:51 (93527): No heartbeat from core client for 30 sec - exiting 15:15:52 (93527): No heartbeat from core client for 30 sec - exiting 15:15:53 (93527): No heartbeat from core client for 30 sec - exiting 15:15:54 (93527): No heartbeat from core client for 30 sec - exiting 23:03:27 (93527): No heartbeat from core client for 30 sec - exiting 23:03:29 (93527): No heartbeat from core client for 30 sec - exiting 23:03:30 (93527): No heartbeat from core client for 30 sec - exiting 23:03:31 (93527): No heartbeat from core client for 30 sec - exiting 23:03:32 (93527): No heartbeat from core client for 30 sec - exiting 23:03:33 (93527): No heartbeat from core client for 30 sec - exiting 23:03:34 (93527): No heartbeat from core client for 30 sec - exiting 23:03:35 (93527): No heartbeat from core client for 30 sec - exiting 23:03:36 (93527): No heartbeat from core client for 30 sec - exiting 23:03:37 (93527): No heartbeat from core client for 30 sec - exiting 23:03:38 (93527): No heartbeat from core client for 30 sec - exiting 23:03:39 (93527): No heartbeat from core client for 30 sec - exiting 23:03:40 (93527): No heartbeat from core client for 30 sec - exiting 23:03:41 (93527): No heartbeat from core client for 30 sec - exiting 23:03:42 (93527): No heartbeat from core client for 30 sec - exiting 23:03:43 (93527): No heartbeat from core client for 30 sec - exiting 23:03:44 (93527): No heartbeat from core client for 30 sec - exiting 05:17:11 (93527): No heartbeat from core client for 30 sec - exiting 05:17:12 (93527): No heartbeat from core client for 30 sec - exiting 05:17:13 (93527): No heartbeat from core client for 30 sec - exiting 05:17:14 (93527): No heartbeat from core client for 30 sec - exiting 05:17:15 (93527): No heartbeat from core client for 30 sec - exiting 05:17:16 (93527): No heartbeat from core client for 30 sec - exiting 05:17:17 (93527): No heartbeat from core client for 30 sec - exiting 05:17:18 (93527): No heartbeat from core client for 30 sec - exiting 05:17:19 (93527): No heartbeat from core client for 30 sec - exiting 15:42:27 (93527): No heartbeat from core client for 30 sec - exiting 15:42:29 (93527): No heartbeat from core client for 30 sec - exiting 15:42:30 (93527): No heartbeat from core client for 30 sec - exiting 15:42:31 (93527): No heartbeat from core client for 30 sec - exiting 15:42:32 (93527): No heartbeat from core client for 30 sec - exiting 15:42:33 (93527): No heartbeat from core client for 30 sec - exiting 15:42:34 (93527): No heartbeat from core client for 30 sec - exiting 15:42:35 (93527): No heartbeat from core client for 30 sec - exiting 16:42:59 (93527): No heartbeat from core client for 30 sec - exiting 16:43:02 (93527): No heartbeat from core client for 30 sec - exiting 16:43:03 (93527): No heartbeat from core client for 30 sec - exiting 16:43:04 (93527): No heartbeat from core client for 30 sec - exiting 16:43:05 (93527): No heartbeat from core client for 30 sec - exiting 16:43:06 (93527): No heartbeat from core client for 30 sec - exiting 16:43:07 (93527): No heartbeat from core client for 30 sec - exiting 16:43:08 (93527): No heartbeat from core client for 30 sec - exiting 16:43:09 (93527): No heartbeat from core client for 30 sec - exiting 16:43:10 (93527): No heartbeat from core client for 30 sec - exiting 16:43:11 (93527): No heartbeat from core client for 30 sec - exiting 16:43:12 (93527): No heartbeat from core client for 30 sec - exiting 16:43:13 (93527): No heartbeat from core client for 30 sec - exiting 16:43:57 (93527): No heartbeat from core client for 30 sec - exiting 16:43:58 (93527): No heartbeat from core client for 30 sec - exiting 16:43:59 (93527): No heartbeat from core client for 30 sec - exiting 16:44:00 (93527): No heartbeat from core client for 30 sec - exiting 16:44:01 (93527): No heartbeat from core client for 30 sec - exiting 16:44:02 (93527): No heartbeat from core client for 30 sec - exiting 16:44:03 (93527): No heartbeat from core client for 30 sec - exiting 16:44:04 (93527): No heartbeat from core client for 30 sec - exiting 16:44:05 (93527): No heartbeat from core client for 30 sec - exiting 16:44:06 (93527): No heartbeat from core client for 30 sec - exiting 16:44:07 (93527): No heartbeat from core client for 30 sec - exiting 16:44:08 (93527): No heartbeat from core client for 30 sec - exiting 16:44:09 (93527): No heartbeat from core client for 30 sec - exiting 16:44:10 (93527): No heartbeat from core client for 30 sec - exiting 16:44:11 (93527): No heartbeat from core client for 30 sec - exiting 16:44:12 (93527): No heartbeat from core client for 30 sec - exiting 16:44:13 (93527): No heartbeat from core client for 30 sec - exiting 16:44:14 (93527): No heartbeat from core client for 30 sec - exiting 16:44:15 (93527): No heartbeat from core client for 30 sec - exiting 16:44:16 (93527): No heartbeat from core client for 30 sec - exiting 16:44:17 (93527): No heartbeat from core client for 30 sec - exiting 16:44:18 (93527): No heartbeat from core client for 30 sec - exiting 16:44:19 (93527): No heartbeat from core client for 30 sec - exiting 16:44:20 (93527): No heartbeat from core client for 30 sec - exiting 16:44:21 (93527): No heartbeat from core client for 30 sec - exiting 16:44:22 (93527): No heartbeat from core client for 30 sec - exiting 16:44:23 (93527): No heartbeat from core client for 30 sec - exiting 16:44:24 (93527): No heartbeat from core client for 30 sec - exiting 16:44:25 (93527): No heartbeat from core client for 30 sec - exiting 16:44:26 (93527): No heartbeat from core client for 30 sec - exiting 16:44:27 (93527): No heartbeat from core client for 30 sec - exiting 16:44:28 (93527): No heartbeat from core client for 30 sec - exiting 16:44:29 (93527): No heartbeat from core client for 30 sec - exiting 16:44:30 (93527): No heartbeat from core client for 30 sec - exiting 16:44:31 (93527): No heartbeat from core client for 30 sec - exiting 16:44:32 (93527): No heartbeat from core client for 30 sec - exiting 16:44:33 (93527): No heartbeat from core client for 30 sec - exiting 16:44:34 (93527): No heartbeat from core client for 30 sec - exiting 16:44:35 (93527): No heartbeat from core client for 30 sec - exiting 16:44:36 (93527): No heartbeat from core client for 30 sec - exiting 16:44:37 (93527): No heartbeat from core client for 30 sec - exiting 16:44:38 (93527): No heartbeat from core client for 30 sec - exiting 16:44:39 (93527): No heartbeat from core client for 30 sec - exiting 16:44:40 (93527): No heartbeat from core client for 30 sec - exiting 16:44:41 (93527): No heartbeat from core client for 30 sec - exiting 16:44:42 (93527): No heartbeat from core client for 30 sec - exiting 16:44:43 (93527): No heartbeat from core client for 30 sec - exiting 16:44:44 (93527): No heartbeat from core client for 30 sec - exiting 16:44:45 (93527): No heartbeat from core client for 30 sec - exiting 16:44:46 (93527): No heartbeat from core client for 30 sec - exiting 16:44:47 (93527): No heartbeat from core client for 30 sec - exiting 16:44:48 (93527): No heartbeat from core client for 30 sec - exiting 16:44:49 (93527): No heartbeat from core client for 30 sec - exiting 16:44:50 (93527): No heartbeat from core client for 30 sec - exiting 16:44:51 (93527): No heartbeat from core client for 30 sec - exiting 16:44:52 (93527): No heartbeat from core client for 30 sec - exiting 16:44:53 (93527): No heartbeat from core client for 30 sec - exiting 16:44:54 (93527): No heartbeat from core client for 30 sec - exiting Signal 1 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x103ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x839004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x103ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x839004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x101ac04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x101ac00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(198,0xa0852540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug 19:20:16 (198): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:17 (198): No heartbeat from core client for 30 sec - exiting 19:20:18 (198): No heartbeat from core client for 30 sec - exiting 19:20:19 (198): No heartbeat from core client for 30 sec - exiting 19:20:20 (198): No heartbeat from core client for 30 sec - exiting 19:20:21 (198): No heartbeat from core client for 30 sec - exiting 19:20:22 (198): No heartbeat from core client for 30 sec - exiting 19:20:23 (198): No heartbeat from core client for 30 sec - exiting 19:20:24 (198): No heartbeat from core client for 30 sec - exiting 19:20:25 (198): No heartbeat from core client for 30 sec - exiting 19:20:26 (198): No heartbeat from core client for 30 sec - exiting 19:20:27 (198): No heartbeat from core client for 30 sec - exiting 19:20:28 (198): No heartbeat from core client for 30 sec - exiting 19:20:29 (198): No heartbeat from core client for 30 sec - exiting 19:20:30 (198): No heartbeat from core client for 30 sec - exiting 19:20:31 (198): No heartbeat from core client for 30 sec - exiting 19:20:32 (198): No heartbeat from core client for 30 sec - exiting 19:20:33 (198): No heartbeat from core client for 30 sec - exiting 19:20:34 (198): No heartbeat from core client for 30 sec - exiting 19:20:35 (198): No heartbeat from core client for 30 sec - exiting 19:20:36 (198): No heartbeat from core client for 30 sec - exiting 19:20:37 (198): No heartbeat from core client for 30 sec - exiting 19:20:38 (198): No heartbeat from core client for 30 sec - exiting 19:20:39 (198): No heartbeat from core client for 30 sec - exiting 19:20:40 (198): No heartbeat from core client for 30 sec - exiting 19:20:41 (198): No heartbeat from core client for 30 sec - exiting 19:20:43 (198): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 16:25:38 (9062): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:40 (9062): No heartbeat from core client for 30 sec - exiting 16:25:41 (9062): No heartbeat from core client for 30 sec - exiting 16:25:42 (9062): No heartbeat from core client for 30 sec - exiting 16:25:43 (9062): No heartbeat from core client for 30 sec - exiting 16:25:44 (9062): No heartbeat from core client for 30 sec - exiting 16:25:45 (9062): No heartbeat from core client for 30 sec - exiting 16:25:46 (9062): No heartbeat from core client for 30 sec - exiting 16:25:47 (9062): No heartbeat from core client for 30 sec - exiting 16:25:48 (9062): No heartbeat from core client for 30 sec - exiting 16:25:49 (9062): No heartbeat from core client for 30 sec - exiting 16:25:50 (9062): No heartbeat from core client for 30 sec - exiting 16:25:51 (9062): No heartbeat from core client for 30 sec - exiting 16:25:52 (9062): No heartbeat from core client for 30 sec - exiting 16:25:53 (9062): No heartbeat from core client for 30 sec - exiting 16:25:54 (9062): No heartbeat from core client for 30 sec - exiting 16:25:55 (9062): No heartbeat from core client for 30 sec - exiting 16:25:56 (9062): No heartbeat from core client for 30 sec - exiting 16:25:57 (9062): No heartbeat from core client for 30 sec - exiting 16:25:59 (9062): No heartbeat from core client for 30 sec - exiting 16:26:00 (9062): No heartbeat from core client for 30 sec - exiting 16:26:01 (9062): No heartbeat from core client for 30 sec - exiting 16:26:02 (9062): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:23:18 (171): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:20 (171): No heartbeat from core client for 30 sec - exiting 17:23:21 (171): No heartbeat from core client for 30 sec - exiting 17:23:22 (171): No heartbeat from core client for 30 sec - exiting 17:23:23 (171): No heartbeat from core client for 30 sec - exiting 17:23:24 (171): No heartbeat from core client for 30 sec - exiting 17:23:25 (171): No heartbeat from core client for 30 sec - exiting 17:23:26 (171): No heartbeat from core client for 30 sec - exiting 17:23:27 (171): No heartbeat from core client for 30 sec - exiting 17:23:28 (171): No heartbeat from core client for 30 sec - exiting 17:23:29 (171): No heartbeat from core client for 30 sec - exiting 17:23:31 (171): No heartbeat from core client for 30 sec - exiting 17:23:32 (171): No heartbeat from core client for 30 sec - exiting 17:23:33 (171): No heartbeat from core client for 30 sec - exiting 17:23:34 (171): No heartbeat from core client for 30 sec - exiting 17:23:35 (171): No heartbeat from core client for 30 sec - exiting 17:23:36 (171): No heartbeat from core client for 30 sec - exiting 17:23:37 (171): No heartbeat from core client for 30 sec - exiting 17:23:38 (171): No heartbeat from core client for 30 sec - exiting 17:23:39 (171): No heartbeat from core client for 30 sec - exiting 17:23:40 (171): No heartbeat from core client for 30 sec - exiting 17:23:41 (171): No heartbeat from core client for 30 sec - exiting 17:23:42 (171): No heartbeat from core client for 30 sec - exiting 17:23:43 (171): No heartbeat from core client for 30 sec - exiting 17:23:44 (171): No heartbeat from core client for 30 sec - exiting 17:23:45 (171): No heartbeat from core client for 30 sec - exiting 17:27:45 (81371): No heartbeat from core client for 30 sec - exiting 17:27:47 (81371): No heartbeat from core client for 30 sec - exiting 17:27:48 (81371): No heartbeat from core client for 30 sec - exiting 17:27:49 (81371): No heartbeat from core client for 30 sec - exiting 17:27:50 (81371): No heartbeat from core client for 30 sec - exiting 17:27:51 (81371): No heartbeat from core client for 30 sec - exiting 17:27:52 (81371): No heartbeat from core client for 30 sec - exiting 17:27:53 (81371): No heartbeat from core client for 30 sec - exiting 17:27:54 (81371): No heartbeat from core client for 30 sec - exiting 17:27:55 (81371): No heartbeat from core client for 30 sec - exiting 17:27:56 (81371): No heartbeat from core client for 30 sec - exiting 17:27:57 (81371): No heartbeat from core client for 30 sec - exiting 17:27:58 (81371): No heartbeat from core client for 30 sec - exiting 17:27:59 (81371): No heartbeat from core client for 30 sec - exiting 17:28:00 (81371): No heartbeat from core client for 30 sec - exiting 17:28:01 (81371): No heartbeat from core client for 30 sec - exiting 17:28:02 (81371): No heartbeat from core client for 30 sec - exiting 17:28:03 (81371): No heartbeat from core client for 30 sec - exiting 17:28:04 (81371): No heartbeat from core client for 30 sec - exiting 17:28:05 (81371): No heartbeat from core client for 30 sec - exiting 17:28:06 (81371): No heartbeat from core client for 30 sec - exiting 17:28:07 (81371): No heartbeat from core client for 30 sec - exiting 17:28:08 (81371): No heartbeat from core client for 30 sec - exiting 17:28:09 (81371): No heartbeat from core client for 30 sec - exiting 17:28:10 (81371): No heartbeat from core client for 30 sec - exiting 17:28:11 (81371): No heartbeat from core client for 30 sec - exiting 17:28:12 (81371): No heartbeat from core client for 30 sec - exiting 17:28:13 (81371): No heartbeat from core client for 30 sec - exiting 17:28:14 (81371): No heartbeat from core client for 30 sec - exiting 17:28:15 (81371): No heartbeat from core client for 30 sec - exiting 17:28:16 (81371): No heartbeat from core client for 30 sec - exiting 17:28:17 (81371): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/203fko.pjg9c10 is not a valid UM file. Error converting file to netcdf: dataout/203fko.pjg9c10 Error: Input file: dataout/203fko.pig9c10 is not a valid UM file. Error converting file to netcdf: dataout/203fko.pig9c10 Error: Input file: dataout/203fko.pfg9c10 is not a valid UM file. Error converting file to netcdf: dataout/203fko.pfg9c10 Error: Input file: dataout/203fka.phg9c10 is not a valid UM file. Error converting file to netcdf: dataout/203fka.phg9c10 Error: Input file: dataout/203fka.pgg9c10 is not a valid UM file. Error converting file to netcdf: dataout/203fka.pgg9c10 Error: Input file: dataout/203fka.peg9c10 is not a valid UM file. Error converting file to netcdf: dataout/203fka.peg9c10 Error: Input file: dataout/203fka.pdg9c10 is not a valid UM file. Error converting file to netcdf: dataout/203fka.pdg9c10 Suspended CPDN Monitor - Suspend request from BOINC... 07:32:12 (81398): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:08:47 (29448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Sep 2012 09:11:56	941153	14935737	hadcm3n_203f_1940_40_007957621_4	1,036,800	1,993,621	1.9229
10 Sep 2012 09:11:56	941153	14935737	hadcm3n_203f_1940_40_007957621_4	1,010,880	1,946,745	1.9258
08 Sep 2012 22:07:21	941153	14935737	hadcm3n_203f_1940_40_007957621_4	984,960	1,899,750	1.9288
08 Sep 2012 22:07:21	941153	14935737	hadcm3n_203f_1940_40_007957621_4	959,040	1,852,816	1.9319
06 Sep 2012 22:21:49	941153	14935737	hadcm3n_203f_1940_40_007957621_4	933,120	1,803,220	1.9325
02 Sep 2012 20:12:32	941153	14935737	hadcm3n_203f_1940_40_007957621_4	907,200	1,751,575	1.9307
01 Sep 2012 17:21:06	941153	14935737	hadcm3n_203f_1940_40_007957621_4	881,280	1,701,663	1.9309
01 Sep 2012 01:23:53	941153	14935737	hadcm3n_203f_1940_40_007957621_4	855,360	1,650,712	1.9298
31 Aug 2012 09:45:40	941153	14935737	hadcm3n_203f_1940_40_007957621_4	829,440	1,599,017	1.9278
30 Aug 2012 20:37:02	941153	14935737	hadcm3n_203f_1940_40_007957621_4	803,520	1,547,499	1.9259
29 Aug 2012 23:18:43	941153	14935737	hadcm3n_203f_1940_40_007957621_4	777,600	1,495,593	1.9233
29 Aug 2012 09:10:30	941153	14935737	hadcm3n_203f_1940_40_007957621_4	751,680	1,443,709	1.9206
27 Aug 2012 23:03:34	941153	14935737	hadcm3n_203f_1940_40_007957621_4	725,760	1,393,583	1.9202
27 Aug 2012 08:46:23	941153	14935737	hadcm3n_203f_1940_40_007957621_4	699,840	1,343,501	1.9197
26 Aug 2012 15:56:35	941153	14935737	hadcm3n_203f_1940_40_007957621_4	673,920	1,293,322	1.9191
25 Aug 2012 14:00:46	941153	14935737	hadcm3n_203f_1940_40_007957621_4	648,000	1,243,083	1.9183
24 Aug 2012 20:56:51	941153	14935737	hadcm3n_203f_1940_40_007957621_4	622,080	1,193,880	1.9192
23 Aug 2012 20:17:30	941153	14935737	hadcm3n_203f_1940_40_007957621_4	596,160	1,143,924	1.9188
13 Aug 2012 08:19:15	941153	14935737	hadcm3n_203f_1940_40_007957621_4	570,240	1,123,333	1.9699
10 Aug 2012 21:19:27	941153	14935737	hadcm3n_203f_1940_40_007957621_4	518,400	990,761	1.9112