Name | hadcm3n_s0w8_1940_40_009094321_0 |
Workunit | 9224657 |
Created | 20 Oct 2014, 17:02:53 UTC |
Sent | 20 Oct 2014, 18:18:36 UTC |
Report deadline | 20 Jan 2015, 1:45:47 UTC |
Received | 22 Mar 2015, 22:40:29 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 1313539 |
Run time | 12 days 21 hours 32 min 3 sec |
CPU time | 10 days 14 hours 50 min 15 sec |
Validate state | Invalid |
Credit | 9,953.28 |
Device peak FLOPS | 2.84 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin |
Stderr | <core_client_version>7.4.36</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> lfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1015c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.10.3 build 14D87p hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1812e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(666,0xa0ea81d4) malloc: *** error for object 0x1812e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug Tue Mar 3 13:48:21 2015 hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x821e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x821e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(545,0xa0ea81d4) malloc: *** error for object 0x280dc04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x100dc04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x100dc00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4805c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4805c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4800e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4800e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4800e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa0ea81d4) malloc: *** error for object 0x4800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.10.3 build 14D87p Thu Mar 5 20:04:31 2015 17:26:16 (668): No heartbeat from core client for 30 sec - exiting 17:26:17 (668): No heartbeat from core client for 30 sec - exiting 17:26:18 (668): No heartbeat from core client for 30 sec - exiting 17:26:19 (668): No heartbeat from core client for 30 sec - exiting 16:23:33 (668): No heartbeat from core client for 30 sec - exiting 16:23:34 (668): No heartbeat from core client for 30 sec - exiting 16:23:35 (668): No heartbeat from core client for 30 sec - exiting 16:23:36 (668): No heartbeat from core client for 30 sec - exiting 16:23:37 (668): No heartbeat from core client for 30 sec - exiting 16:23:38 (668): No heartbeat from core client for 30 sec - exiting 16:23:39 (668): No heartbeat from core client for 30 sec - exiting 16:23:40 (668): No heartbeat from core client for 30 sec - exiting 16:23:41 (668): No heartbeat from core client for 30 sec - exiting 16:23:42 (668): No heartbeat from core client for 30 sec - exiting 16:23:43 (668): No heartbeat from core client for 30 sec - exiting 16:23:44 (668): No heartbeat from core client for 30 sec - exiting 16:23:45 (668): No heartbeat from core client for 30 sec - exiting 16:23:46 (668): No heartbeat from core client for 30 sec - exiting 16:23:47 (668): No heartbeat from core client for 30 sec - exiting 16:23:48 (668): No heartbeat from core client for 30 sec - exiting 16:23:49 (668): No heartbeat from core client for 30 sec - exiting 16:23:50 (668): No heartbeat from core client for 30 sec - exiting 16:23:51 (668): No heartbeat from core client for 30 sec - exiting 16:23:52 (668): No heartbeat from core client for 30 sec - exiting 16:23:53 (668): No heartbeat from core client for 30 sec - exiting 16:23:54 (668): No heartbeat from core client for 30 sec - exiting 16:23:55 (668): No heartbeat from core client for 30 sec - exiting 16:23:56 (668): No heartbeat from core client for 30 sec - exiting 16:23:57 (668): No heartbeat from core client for 30 sec - exiting 16:23:58 (668): No heartbeat from core client for 30 sec - exiting 16:23:59 (668): No heartbeat from core client for 30 sec - exiting 16:24:00 (668): No heartbeat from core client for 30 sec - exiting 16:24:01 (668): No heartbeat from core client for 30 sec - exiting 16:24:02 (668): No heartbeat from core client for 30 sec - exiting 16:24:03 (668): No heartbeat from core client for 30 sec - exiting 16:24:04 (668): No heartbeat from core client for 30 sec - exiting 16:24:05 (668): No heartbeat from core client for 30 sec - exiting 16:24:07 (668): No heartbeat from core client for 30 sec - exiting 16:24:08 (668): No heartbeat from core client for 30 sec - exiting 16:24:09 (668): No heartbeat from core client for 30 sec - exiting 16:24:10 (668): No heartbeat from core client for 30 sec - exiting 16:24:11 (668): No heartbeat from core client for 30 sec - exiting 16:24:12 (668): No heartbeat from core client for 30 sec - exiting 16:24:13 (668): No heartbeat from core client for 30 sec - exiting 16:24:14 (668): No heartbeat from core client for 30 sec - exiting 16:24:15 (668): No heartbeat from core client for 30 sec - exiting 16:24:16 (668): No heartbeat from core client for 30 sec - exiting 16:24:17 (668): No heartbeat from core client for 30 sec - exiting 16:24:18 (668): No heartbeat from core client for 30 sec - exiting 16:24:19 (668): No heartbeat from core client for 30 sec - exiting 16:24:20 (668): No heartbeat from core client for 30 sec - exiting 16:24:21 (668): No heartbeat from core client for 30 sec - exiting 16:24:22 (668): No heartbeat from core client for 30 sec - exiting 16:24:23 (668): No heartbeat from core client for 30 sec - exiting 16:24:24 (668): No heartbeat from core client for 30 sec - exiting 16:24:25 (668): No heartbeat from core client for 30 sec - exiting 16:24:26 (668): No heartbeat from core client for 30 sec - exiting 16:24:27 (668): No heartbeat from core client for 30 sec - exiting 16:24:28 (668): No heartbeat from core client for 30 sec - exiting 16:24:29 (668): No heartbeat from core client for 30 sec - exiting 16:24:30 (668): No heartbeat from core client for 30 sec - exiting 16:24:31 (668): No heartbeat from core client for 30 sec - exiting 16:24:32 (668): No heartbeat from core client for 30 sec - exiting 16:24:33 (668): No heartbeat from core client for 30 sec - exiting 16:24:34 (668): No heartbeat from core client for 30 sec - exiting 16:24:35 (668): No heartbeat from core client for 30 sec - exiting 16:24:36 (668): No heartbeat from core client for 30 sec - exiting 16:24:37 (668): No heartbeat from core client for 30 sec - exiting 16:24:38 (668): No heartbeat from core client for 30 sec - exiting 16:24:39 (668): No heartbeat from core client for 30 sec - exiting 16:24:40 (668): No heartbeat from core client for 30 sec - exiting hadcm3n_6.07_i686-apple-darwin(668,0xa114b1d4) malloc: *** error for object 0x1015e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa114b1d4) malloc: *** error for object 0x1015e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa114b1d4) malloc: *** error for object 0x3801c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa114b1d4) malloc: *** error for object 0x3801c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa114b1d4) malloc: *** error for object 0x3801c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa114b1d4) malloc: *** error for object 0x3801c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(668,0xa114b1d4) malloc: *** error for object 0x809e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation 17:20:55 (668): No heartbeat from core client for 30 sec - exiting 17:20:56 (668): No heartbeat from core client for 30 sec - exiting 17:20:57 (668): No heartbeat from core client for 30 sec - exiting 17:20:58 (668): No heartbeat from core client for 30 sec - exiting 17:20:59 (668): No heartbeat from core client for 30 sec - exiting 17:21:00 (668): No heartbeat from core client for 30 sec - exiting 17:21:01 (668): No heartbeat from core client for 30 sec - exiting 17:21:02 (668): No heartbeat from core client for 30 sec - exiting 17:21:03 (668): No heartbeat from core client for 30 sec - exiting 17:21:04 (668): No heartbeat from core client for 30 sec - exiting 17:21:05 (668): No heartbeat from core client for 30 sec - exiting 17:21:06 (668): No heartbeat from core client for 30 sec - exiting 17:21:07 (668): No heartbeat from core client for 30 sec - exiting 17:21:08 (668): No heartbeat from core client for 30 sec - exiting 17:21:09 (668): No heartbeat from core client for 30 sec - exiting 17:21:10 (668): No heartbeat from core client for 30 sec - exiting 17:21:12 (668): No heartbeat from core client for 30 sec - exiting 17:21:13 (668): No heartbeat from core client for 30 sec - exiting 17:21:14 (668): No heartbeat from core client for 30 sec - exiting 17:21:15 (668): No heartbeat from core client for 30 sec - exiting 17:21:16 (668): No heartbeat from core client for 30 sec - exiting 17:21:17 (668): No heartbeat from core client for 30 sec - exiting 17:21:18 (668): No heartbeat from core client for 30 sec - exiting 17:21:19 (668): No heartbeat from core client for 30 sec - exiting 17:21:20 (668): No heartbeat from core client for 30 sec - exiting 17:21:21 (668): No heartbeat from core client for 30 sec - exiting 17:21:22 (668): No heartbeat from core client for 30 sec - exiting 17:21:23 (668): No heartbeat from core client for 30 sec - exiting 17:21:24 (668): No heartbeat from core client for 30 sec - exiting 17:21:25 (668): No heartbeat from core client for 30 sec - exiting 17:21:26 (668): No heartbeat from core client for 30 sec - exiting 17:21:27 (668): No heartbeat from core client for 30 sec - exiting 08:40:32 (668): No heartbeat from core client for 30 sec - exiting 08:40:33 (668): No heartbeat from core client for 30 sec - exiting 08:40:34 (668): No heartbeat from core client for 30 sec - exiting 08:40:35 (668): No heartbeat from core client for 30 sec - exiting 08:40:36 (668): No heartbeat from core client for 30 sec - exiting 08:40:37 (668): No heartbeat from core client for 30 sec - exiting 08:40:38 (668): No heartbeat from core client for 30 sec - exiting 08:40:39 (668): No heartbeat from core client for 30 sec - exiting 08:40:40 (668): No heartbeat from core client for 30 sec - exiting 08:40:41 (668): No heartbeat from core client for 30 sec - exiting 08:40:42 (668): No heartbeat from core client for 30 sec - exiting 08:40:43 (668): No heartbeat from core client for 30 sec - exiting 08:40:44 (668): No heartbeat from core client for 30 sec - exiting 08:40:45 (668): No heartbeat from core client for 30 sec - exiting 08:40:46 (668): No heartbeat from core client for 30 sec - exiting 08:40:47 (668): No heartbeat from core client for 30 sec - exiting 08:40:48 (668): No heartbeat from core client for 30 sec - exiting 08:40:50 (668): No heartbeat from core client for 30 sec - exiting 08:40:51 (668): No heartbeat from core client for 30 sec - exiting 08:40:52 (668): No heartbeat from core client for 30 sec - exiting 08:40:53 (668): No heartbeat from core client for 30 sec - exiting 08:40:54 (668): No heartbeat from core client for 30 sec - exiting 08:40:55 (668): No heartbeat from core client for 30 sec - exiting 08:40:56 (668): No heartbeat from core client for 30 sec - exiting 08:40:57 (668): No heartbeat from core client for 30 sec - exiting 08:40:58 (668): No heartbeat from core client for 30 sec - exiting 08:40:59 (668): No heartbeat from core client for 30 sec - exiting 08:41:00 (668): No heartbeat from core client for 30 sec - exiting 08:41:01 (668): No heartbeat from core client for 30 sec - exiting 08:41:02 (668): No heartbeat from core client for 30 sec - exiting 08:41:03 (668): No heartbeat from core client for 30 sec - exiting 08:41:04 (668): No heartbeat from core client for 30 sec - exiting 08:41:05 (668): No heartbeat from core client for 30 sec - exiting 08:41:06 (668): No heartbeat from core client for 30 sec - exiting 08:41:07 (668): No heartbeat from core client for 30 sec - exiting 08:41:08 (668): No heartbeat from core client for 30 sec - exiting 08:41:09 (668): No heartbeat from core client for 30 sec - exiting 08:41:10 (668): No heartbeat from core client for 30 sec - exiting 08:41:11 (668): No heartbeat from core client for 30 sec - exiting 08:41:12 (668): No heartbeat from core client for 30 sec - exiting 08:41:13 (668): No heartbeat from core client for 30 sec - exiting 08:41:14 (668): No heartbeat from core client for 30 sec - exiting 08:41:16 (668): No heartbeat from core client for 30 sec - exiting 08:41:17 (668): No heartbeat from core client for 30 sec - exiting 08:41:18 (668): No heartbeat from core client for 30 sec - exiting 08:41:19 (668): No heartbeat from core client for 30 sec - exiting 08:41:20 (668): No heartbeat from core client for 30 sec - exiting 08:41:21 (668): No heartbeat from core client for 30 sec - exiting 08:41:22 (668): No heartbeat from core client for 30 sec - exiting 08:41:23 (668): No heartbeat from core client for 30 sec - exiting 08:41:24 (668): No heartbeat from core client for 30 sec - exiting 08:41:25 (668): No heartbeat from core client for 30 sec - exiting 08:41:26 (668): No heartbeat from core client for 30 sec - exiting 08:41:27 (668): No heartbeat from core client for 30 sec - exiting 08:41:28 (668): No heartbeat from core client for 30 sec - exiting 08:41:29 (668): No heartbeat from core client for 30 sec - exiting 08:41:30 (668): No heartbeat from core client for 30 sec - exiting 08:41:31 (668): No heartbeat from core client for 30 sec - exiting 08:41:32 (668): No heartbeat from core client for 30 sec - exiting 08:41:33 (668): No heartbeat from core client for 30 sec - exiting 08:41:34 (668): No heartbeat from core client for 30 sec - exiting 08:41:35 (668): No heartbeat from core client for 30 sec - exiting 08:41:36 (668): No heartbeat from core client for 30 sec - exiting 08:41:37 (668): No heartbeat from core client for 30 sec - exiting 08:41:39 (668): No heartbeat from core client for 30 sec - exiting 08:41:40 (668): No heartbeat from core client for 30 sec - exiting 08:41:41 (668): No heartbeat from core client for 30 sec - exiting 08:41:42 (668): No heartbeat from core client for 30 sec - exiting 08:41:43 (668): No heartbeat from core client for 30 sec - exiting 08:41:44 (668): No heartbeat from core client for 30 sec - exiting 08:41:45 (668): No heartbeat from core client for 30 sec - exiting 08:41:46 (668): No heartbeat from core client for 30 sec - exiting 08:41:47 (668): No heartbeat from core client for 30 sec - exiting 08:41:48 (668): No heartbeat from core client for 30 sec - exiting 08:41:49 (668): No heartbeat from core client for 30 sec - exiting 08:41:50 (668): No heartbeat from core client for 30 sec - exiting 08:41:51 (668): No heartbeat from core client for 30 sec - exiting 08:41:52 (668): No heartbeat from core client for 30 sec - exiting 08:41:53 (668): No heartbeat from core client for 30 sec - exiting 08:41:54 (668): No heartbeat from core client for 30 sec - exiting 08:41:55 (668): No heartbeat from core client for 30 sec - exiting 08:41:56 (668): No heartbeat from core client for 30 sec - exiting 08:41:57 (668): No heartbeat from core client for 30 sec - exiting 08:41:58 (668): No heartbeat from core client for 30 sec - exiting 08:41:59 (668): No heartbeat from core client for 30 sec - exiting 08:42:00 (668): No heartbeat from core client for 30 sec - exiting 08:42:01 (668): No heartbeat from core client for 30 sec - exiting 08:42:02 (668): No heartbeat from core client for 30 sec - exiting 08:42:03 (668): No heartbeat from core client for 30 sec - exiting 08:42:05 (668): No heartbeat from core client for 30 sec - exiting 08:42:06 (668): No heartbeat from core client for 30 sec - exiting 08:42:07 (668): No heartbeat from core client for 30 sec - exiting 08:42:08 (668): No heartbeat from core client for 30 sec - exiting 08:42:09 (668): No heartbeat from core client for 30 sec - exiting 08:42:10 (668): No heartbeat from core client for 30 sec - exiting 08:42:11 (668): No heartbeat from core client for 30 sec - exiting 08:42:12 (668): No heartbeat from core client for 30 sec - exiting 08:42:13 (668): No heartbeat from core client for 30 sec - exiting 08:42:14 (668): No heartbeat from core client for 30 sec - exiting 08:42:15 (668): No heartbeat from core client for 30 sec - exiting 08:42:16 (668): No heartbeat from core client for 30 sec - exiting 08:42:17 (668): No heartbeat from core client for 30 sec - exiting 08:42:18 (668): No heartbeat from core client for 30 sec - exiting 08:42:19 (668): No heartbeat from core client for 30 sec - exiting 08:42:20 (668): No heartbeat from core client for 30 sec - exiting 08:42:21 (668): No heartbeat from core client for 30 sec - exiting 08:42:22 (668): No heartbeat from core client for 30 sec - exiting 08:42:24 (668): No heartbeat from core client for 30 sec - exiting 08:42:25 (668): No heartbeat from core client for 30 sec - exiting 08:42:26 (668): No heartbeat from core client for 30 sec - exiting 08:42:27 (668): No heartbeat from core client for 30 sec - exiting 08:42:28 (668): No heartbeat from core client for 30 sec - exiting 08:42:29 (668): No heartbeat from core client for 30 sec - exiting 08:42:30 (668): No heartbeat from core client for 30 sec - exiting 08:42:31 (668): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x1005e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x1005e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x2031404: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x2031400: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x3000e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x3000e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x3000e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x3800e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(860,0xa06b61d4) malloc: *** error for object 0x3800e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.10.3 build 14D105g Tue Mar 17 08:48:30 2015 08:48:59 (860): No heartbeat from core client for 30 sec - exiting 08:49:00 (860): No heartbeat from core client for 30 sec - exiting 08:49:01 (860): No heartbeat from core client for 30 sec - exiting 08:49:02 (860): No heartbeat from core client for 30 sec - exiting 08:49:03 (860): No heartbeat from core client for 30 sec - exiting 08:49:04 (860): No heartbeat from core client for 30 sec - exiting 08:49:05 (860): No heartbeat from core client for 30 sec - exiting 08:49:06 (860): No heartbeat from core client for 30 sec - exiting 08:49:07 (860): No heartbeat from core client for 30 sec - exiting 08:49:08 (860): No heartbeat from core client for 30 sec - exiting 08:49:10 (860): No heartbeat from core client for 30 sec - exiting 08:49:11 (860): No heartbeat from core client for 30 sec - exiting 08:49:12 (860): No heartbeat from core client for 30 sec - exiting 08:49:13 (860): No heartbeat from core client for 30 sec - exiting 08:49:14 (860): No heartbeat from core client for 30 sec - exiting 08:49:15 (860): No heartbeat from core client for 30 sec - exiting 08:49:16 (860): No heartbeat from core client for 30 sec - exiting 08:49:17 (860): No heartbeat from core client for 30 sec - exiting 08:49:18 (860): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x281ac04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x281ac00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x401d200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2612,0xa05d81d4) malloc: *** error for object 0x1004e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.10.3 build 14D105g Thu Mar 19 09:53:33 2015 19:07:04 (2612): No heartbeat from core client for 30 sec - exiting 19:07:05 (2612): No heartbeat from core client for 30 sec - exiting 19:07:06 (2612): No heartbeat from core client for 30 sec - exiting 19:07:07 (2612): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x2005c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x2005c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x303c204: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x303c200: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x2005c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x2005c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x2005c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x2005c00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(2362,0xa05d81d4) malloc: *** error for object 0x2005c04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e00: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(713,0xa05d81d4) malloc: *** error for object 0x819e04: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.10.3 build 14D105g Sun Mar 22 18:39:06 2015 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fd68 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/0/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90008000 - 0x90008fff /usr/lib/system/liblaunch.dylib 0x9005d000 - 0x9005dfff /usr/lib/system/libunc.dylib 0x9034f000 - 0x9036efff /usr/lib/system/libsystem_kernel.dylib 0x907ba000 - 0x907c3fff /usr/lib/system/libcopyfile.dylib 0x90d9b000 - 0x90d9efff /usr/lib/system/libdyld.dylib 0x90d9f000 - 0x90da4fff /usr/lib/system/libcompiler_rt.dylib 0x90db3000 - 0x90db9fff /usr/lib/system/libsystem_platform.dylib 0x9183c000 - 0x91855fff /usr/lib/system/libsystem_malloc.dylib 0x91896000 - 0x9189efff /usr/lib/system/libsystem_pthread.dylib 0x91f07000 - 0x91f13fff /usr/lib/system/libcommonCrypto.dylib 0x925ac000 - 0x925c3fff /usr/lib/system/libsystem_asl.dylib 0x925c4000 - 0x92659fff /usr/lib/system/libsystem_c.dylib 0x931d5000 - 0x931defff /usr/lib/system/libsystem_notify.dylib 0x9346b000 - 0x93472fff /usr/lib/system/libunwind.dylib 0x956ca000 - 0x956f3fff /usr/lib/system/libsystem_info.dylib 0x95736000 - 0x9575cfff /usr/lib/libc++abi.dylib 0x9589a000 - 0x958edfff /usr/lib/libstdc++.6.dylib 0x96785000 - 0x967b8fff /usr/lib/system/libsystem_m.dylib 0x97c4e000 - 0x97c54fff /usr/lib/system/libsystem_networkextension.dylib 0x97d5a000 - 0x97d5afff /usr/lib/system/libkeymgr.dylib 0x97d5b000 - 0x97d6cfff /usr/lib/system/libsystem_coretls.dylib 0x986b1000 - 0x986b3fff /usr/lib/system/libsystem_coreservices.dylib 0x986b4000 - 0x986dbfff /usr/lib/system/libdispatch.dylib 0x98c97000 - 0x98c99fff /usr/lib/system/libsystem_configuration.dylib 0x98c9a000 - 0x98c9bfff /usr/lib/libSystem.B.dylib 0x98f8b000 - 0x98f8dfff /usr/lib/system/libquarantine.dylib 0x993ee000 - 0x993f2fff /usr/lib/system/libcache.dylib 0x997b8000 - 0x997bafff /usr/lib/system/libsystem_sandbox.dylib 0x99987000 - 0x999adfff /usr/lib/system/libxpc.dylib 0x99d35000 - 0x99d3bfff /usr/lib/system/libsystem_trace.dylib 0x99d3d000 - 0x99d3efff /usr/lib/system/libremovefile.dylib 0x99eee000 - 0x99efcfff /usr/lib/libz.1.dylib 0x9a9eb000 - 0x9aa28fff /usr/lib/system/libsystem_network.dylib 0x9aa29000 - 0x9aa2efff /usr/lib/system/libmacho.dylib 0x9aaf0000 - 0x9aaf2fff /usr/lib/system/libsystem_secinit.dylib 0x9b222000 - 0x9b22afff /usr/lib/system/libsystem_dnssd.dylib 0x9b22b000 - 0x9b22cfff /usr/lib/system/libsystem_blocks.dylib 0x9bcc9000 - 0x9bd32fff /usr/lib/system/libcorecrypto.dylib Exiting... </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
03 Mar 2015 18:51:53 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 829,440 | 917,415 | 1.1061 |
28 Feb 2015 21:13:09 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 777,600 | 877,269 | 1.1282 |
26 Feb 2015 18:11:48 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 751,680 | 859,781 | 1.1438 |
17 Feb 2015 22:57:32 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 725,760 | 842,848 | 1.1613 |
16 Feb 2015 02:17:16 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 699,840 | 826,253 | 1.1806 |
12 Feb 2015 07:46:35 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 673,920 | 804,517 | 1.1938 |
09 Feb 2015 00:47:43 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 648,000 | 785,683 | 1.2125 |
07 Feb 2015 00:24:18 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 622,080 | 767,998 | 1.2346 |
04 Feb 2015 03:43:27 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 596,160 | 751,143 | 1.2600 |
01 Feb 2015 01:04:37 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 570,240 | 735,670 | 1.2901 |
31 Jan 2015 02:44:16 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 544,320 | 719,668 | 1.3221 |
30 Jan 2015 08:33:59 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 518,400 | 697,021 | 1.3446 |
29 Jan 2015 01:29:02 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 492,480 | 681,317 | 1.3834 |
22 Jan 2015 00:38:21 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 466,560 | 670,404 | 1.4369 |
18 Jan 2015 02:21:18 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 440,640 | 633,770 | 1.4383 |
12 Jan 2015 18:35:23 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 414,720 | 596,916 | 1.4393 |
04 Jan 2015 23:00:51 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 388,800 | 559,855 | 1.4400 |
04 Jan 2015 06:27:50 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 362,880 | 523,517 | 1.4427 |
01 Jan 2015 21:19:37 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 336,960 | 486,448 | 1.4436 |
28 Dec 2014 00:06:13 | 1313539 | 17229992 | hadcm3n_s0w8_1940_40_009094321_0 | 311,040 | 448,751 | 1.4427 |
©2024 climateprediction.net