Task 16389942

Name	hadcm3n_85vq_1980_40_008514125_2
Workunit	8661637
Created	24 Mar 2014, 10:00:51 UTC
Sent	24 Mar 2014, 10:01:06 UTC
Report deadline	23 Jun 2014, 17:28:17 UTC
Received	13 May 2014, 6:57:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1143449
Run time	19 days 6 hours 32 min 19 sec
CPU time	17 days 18 hours 46 min 14 sec
Validate state	Invalid
Credit	9,642.24
Device peak FLOPS	2.95 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> ore client for 30 sec - exiting 09:51:39 (908): No heartbeat from core client for 30 sec - exiting 09:51:40 (908): No heartbeat from core client for 30 sec - exiting 09:51:42 (908): No heartbeat from core client for 30 sec - exiting 09:51:43 (908): No heartbeat from core client for 30 sec - exiting 09:51:44 (908): No heartbeat from core client for 30 sec - exiting 09:51:45 (908): No heartbeat from core client for 30 sec - exiting 09:51:46 (908): No heartbeat from core client for 30 sec - exiting 09:51:47 (908): No heartbeat from core client for 30 sec - exiting 09:51:48 (908): No heartbeat from core client for 30 sec - exiting 09:51:49 (908): No heartbeat from core client for 30 sec - exiting 09:51:50 (908): No heartbeat from core client for 30 sec - exiting 09:51:51 (908): No heartbeat from core client for 30 sec - exiting 09:51:52 (908): No heartbeat from core client for 30 sec - exiting 09:51:54 (908): No heartbeat from core client for 30 sec - exiting 09:51:55 (908): No heartbeat from core client for 30 sec - exiting 09:51:56 (908): No heartbeat from core client for 30 sec - exiting 09:51:57 (908): No heartbeat from core client for 30 sec - exiting 09:51:58 (908): No heartbeat from core client for 30 sec - exiting 09:51:59 (908): No heartbeat from core client for 30 sec - exiting 09:52:00 (908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7348, iMonCtr=1 Model crash detected, will try to restart... 10:40:23 (6524): No heartbeat from core client for 30 sec - exiting 10:40:24 (6524): No heartbeat from core client for 30 sec - exiting 10:40:25 (6524): No heartbeat from core client for 30 sec - exiting 10:40:26 (6524): No heartbeat from core client for 30 sec - exiting 10:40:28 (6524): No heartbeat from core client for 30 sec - exiting 10:40:29 (6524): No heartbeat from core client for 30 sec - exiting 10:40:30 (6524): No heartbeat from core client for 30 sec - exiting 10:40:31 (6524): No heartbeat from core client for 30 sec - exiting 10:40:32 (6524): No heartbeat from core client for 30 sec - exiting 10:40:33 (6524): No heartbeat from core client for 30 sec - exiting 10:40:34 (6524): No heartbeat from core client for 30 sec - exiting 10:40:35 (6524): No heartbeat from core client for 30 sec - exiting 10:40:36 (6524): No heartbeat from core client for 30 sec - exiting 10:40:37 (6524): No heartbeat from core client for 30 sec - exiting 10:40:38 (6524): No heartbeat from core client for 30 sec - exiting 10:40:40 (6524): No heartbeat from core client for 30 sec - exiting 10:40:41 (6524): No heartbeat from core client for 30 sec - exiting 10:40:42 (6524): No heartbeat from core client for 30 sec - exiting 10:40:43 (6524): No heartbeat from core client for 30 sec - exiting 10:40:44 (6524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7520, iMonCtr=1 Model crash detected, will try to restart... 10:26:39 (5724): No heartbeat from core client for 30 sec - exiting 10:26:40 (5724): No heartbeat from core client for 30 sec - exiting 10:26:41 (5724): No heartbeat from core client for 30 sec - exiting 10:26:42 (5724): No heartbeat from core client for 30 sec - exiting 10:26:43 (5724): No heartbeat from core client for 30 sec - exiting 10:26:44 (5724): No heartbeat from core client for 30 sec - exiting 10:26:45 (5724): No heartbeat from core client for 30 sec - exiting 10:26:46 (5724): No heartbeat from core client for 30 sec - exiting 10:26:47 (5724): No heartbeat from core client for 30 sec - exiting 10:26:48 (5724): No heartbeat from core client for 30 sec - exiting 10:26:50 (5724): No heartbeat from core client for 30 sec - exiting 10:26:51 (5724): No heartbeat from core client for 30 sec - exiting 10:26:52 (5724): No heartbeat from core client for 30 sec - exiting 10:26:53 (5724): No heartbeat from core client for 30 sec - exiting 10:26:54 (5724): No heartbeat from core client for 30 sec - exiting 10:26:55 (5724): No heartbeat from core client for 30 sec - exiting 10:26:56 (5724): No heartbeat from core client for 30 sec - exiting 10:26:57 (5724): No heartbeat from core client for 30 sec - exiting 10:26:58 (5724): No heartbeat from core client for 30 sec - exiting 10:26:59 (5724): No heartbeat from core client for 30 sec - exiting 10:27:00 (5724): No heartbeat from core client for 30 sec - exiting 10:27:02 (5724): No heartbeat from core client for 30 sec - exiting 10:27:03 (5724): No heartbeat from core client for 30 sec - exiting 10:27:04 (5724): No heartbeat from core client for 30 sec - exiting 10:27:05 (5724): No heartbeat from core client for 30 sec - exiting 10:27:06 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=908, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7316, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7316, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7316, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6688, iMonCtr=1 Model crash detected, will try to restart... 11:11:57 (6260): No heartbeat from core client for 30 sec - exiting 11:11:58 (6260): No heartbeat from core client for 30 sec - exiting 11:11:59 (6260): No heartbeat from core client for 30 sec - exiting 11:12:00 (6260): No heartbeat from core client for 30 sec - exiting 11:12:01 (6260): No heartbeat from core client for 30 sec - exiting 11:12:02 (6260): No heartbeat from core client for 30 sec - exiting 11:12:03 (6260): No heartbeat from core client for 30 sec - exiting 11:12:04 (6260): No heartbeat from core client for 30 sec - exiting 11:12:06 (6260): No heartbeat from core client for 30 sec - exiting 11:12:07 (6260): No heartbeat from core client for 30 sec - exiting 11:12:08 (6260): No heartbeat from core client for 30 sec - exiting 11:12:09 (6260): No heartbeat from core client for 30 sec - exiting 11:12:10 (6260): No heartbeat from core client for 30 sec - exiting 11:12:11 (6260): No heartbeat from core client for 30 sec - exiting 11:12:12 (6260): No heartbeat from core client for 30 sec - exiting 11:12:13 (6260): No heartbeat from core client for 30 sec - exiting 11:12:14 (6260): No heartbeat from core client for 30 sec - exiting 11:12:15 (6260): No heartbeat from core client for 30 sec - exiting 11:12:16 (6260): No heartbeat from core client for 30 sec - exiting 11:12:18 (6260): No heartbeat from core client for 30 sec - exiting 11:12:19 (6260): No heartbeat from core client for 30 sec - exiting 11:12:20 (6260): No heartbeat from core client for 30 sec - exiting 11:12:21 (6260): No heartbeat from core client for 30 sec - exiting 11:12:22 (6260): No heartbeat from core client for 30 sec - exiting 11:12:23 (6260): No heartbeat from core client for 30 sec - exiting 11:12:24 (6260): No heartbeat from core client for 30 sec - exiting 11:12:25 (6260): No heartbeat from core client for 30 sec - exiting 11:12:26 (6260): No heartbeat from core client for 30 sec - exiting 11:12:27 (6260): No heartbeat from core client for 30 sec - exiting 11:12:29 (6260): No heartbeat from core client for 30 sec - exiting 11:12:30 (6260): No heartbeat from core client for 30 sec - exiting 11:12:31 (6260): No heartbeat from core client for 30 sec - exiting 11:12:32 (6260): No heartbeat from core client for 30 sec - exiting 11:12:33 (6260): No heartbeat from core client for 30 sec - exiting 11:12:34 (6260): No heartbeat from core client for 30 sec - exiting 11:12:35 (6260): No heartbeat from core client for 30 sec - exiting 11:12:36 (6260): No heartbeat from core client for 30 sec - exiting 11:12:37 (6260): No heartbeat from core client for 30 sec - exiting 11:12:38 (6260): No heartbeat from core client for 30 sec - exiting 11:12:39 (6260): No heartbeat from core client for 30 sec - exiting 11:12:41 (6260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6488, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5696, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1592, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6656, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6316, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6544, iMonCtr=1 Model crash detected, will try to restart... 10:17:43 (7872): No heartbeat from core client for 30 sec - exiting 10:17:44 (7872): No heartbeat from core client for 30 sec - exiting 10:17:45 (7872): No heartbeat from core client for 30 sec - exiting 10:17:47 (7872): No heartbeat from core client for 30 sec - exiting 10:17:48 (7872): No heartbeat from core client for 30 sec - exiting 10:17:49 (7872): No heartbeat from core client for 30 sec - exiting 10:17:50 (7872): No heartbeat from core client for 30 sec - exiting 10:17:51 (7872): No heartbeat from core client for 30 sec - exiting 10:17:52 (7872): No heartbeat from core client for 30 sec - exiting 10:17:53 (7872): No heartbeat from core client for 30 sec - exiting 10:17:54 (7872): No heartbeat from core client for 30 sec - exiting 10:17:55 (7872): No heartbeat from core client for 30 sec - exiting 10:17:56 (7872): No heartbeat from core client for 30 sec - exiting 10:17:58 (7872): No heartbeat from core client for 30 sec - exiting 10:17:59 (7872): No heartbeat from core client for 30 sec - exiting 10:18:00 (7872): No heartbeat from core client for 30 sec - exiting 10:18:01 (7872): No heartbeat from core client for 30 sec - exiting 10:18:02 (7872): No heartbeat from core client for 30 sec - exiting 10:18:03 (7872): No heartbeat from core client for 30 sec - exiting 10:18:04 (7872): No heartbeat from core client for 30 sec - exiting 10:18:05 (7872): No heartbeat from core client for 30 sec - exiting 10:18:06 (7872): No heartbeat from core client for 30 sec - exiting 10:18:07 (7872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5068, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:22:26 (6768): No heartbeat from core client for 30 sec - exiting 10:22:27 (6768): No heartbeat from core client for 30 sec - exiting 10:22:28 (6768): No heartbeat from core client for 30 sec - exiting 10:22:29 (6768): No heartbeat from core client for 30 sec - exiting 10:22:30 (6768): No heartbeat from core client for 30 sec - exiting 10:22:31 (6768): No heartbeat from core client for 30 sec - exiting 10:22:32 (6768): No heartbeat from core client for 30 sec - exiting 10:22:33 (6768): No heartbeat from core client for 30 sec - exiting 10:22:34 (6768): No heartbeat from core client for 30 sec - exiting 10:22:36 (6768): No heartbeat from core client for 30 sec - exiting 10:22:37 (6768): No heartbeat from core client for 30 sec - exiting 10:22:38 (6768): No heartbeat from core client for 30 sec - exiting 10:22:39 (6768): No heartbeat from core client for 30 sec - exiting 10:22:40 (6768): No heartbeat from core client for 30 sec - exiting 10:22:41 (6768): No heartbeat from core client for 30 sec - exiting 10:22:42 (6768): No heartbeat from core client for 30 sec - exiting 10:22:43 (6768): No heartbeat from core client for 30 sec - exiting 10:22:44 (6768): No heartbeat from core client for 30 sec - exiting 10:22:45 (6768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:46 (6768): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4416, iMonCtr=1 Model crash detected, will try to restart... 09:42:38 (8124): No heartbeat from core client for 30 sec - exiting 09:42:39 (8124): No heartbeat from core client for 30 sec - exiting 09:42:40 (8124): No heartbeat from core client for 30 sec - exiting 09:42:42 (8124): No heartbeat from core client for 30 sec - exiting 09:42:43 (8124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7872, iMonCtr=1 Model crash detected, will try to restart... 13:59:55 (7464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2272, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7036, iMonCtr=1 Model crash detected, will try to restart... 09:39:31 (1164): No heartbeat from core client for 30 sec - exiting 09:39:32 (1164): No heartbeat from core client for 30 sec - exiting 09:39:33 (1164): No heartbeat from core client for 30 sec - exiting 09:39:34 (1164): No heartbeat from core client for 30 sec - exiting 09:39:35 (1164): No heartbeat from core client for 30 sec - exiting 09:39:36 (1164): No heartbeat from core client for 30 sec - exiting 09:39:37 (1164): No heartbeat from core client for 30 sec - exiting 09:39:38 (1164): No heartbeat from core client for 30 sec - exiting 09:39:39 (1164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3116, iMonCtr=1 Model crash detected, will try to restart... 09:52:09 (4624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5920, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4208, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4652, iMonCtr=1 Model crash detected, will try to restart... 12:20:27 (6356): No heartbeat from core client for 30 sec - exiting 12:20:28 (6356): No heartbeat from core client for 30 sec - exiting 12:20:29 (6356): No heartbeat from core client for 30 sec - exiting 12:20:30 (6356): No heartbeat from core client for 30 sec - exiting 12:20:32 (6356): No heartbeat from core client for 30 sec - exiting 12:20:33 (6356): No heartbeat from core client for 30 sec - exiting 12:20:34 (6356): No heartbeat from core client for 30 sec - exiting 12:20:35 (6356): No heartbeat from core client for 30 sec - exiting 12:20:36 (6356): No heartbeat from core client for 30 sec - exiting 12:20:37 (6356): No heartbeat from core client for 30 sec - exiting 12:20:38 (6356): No heartbeat from core client for 30 sec - exiting 12:20:39 (6356): No heartbeat from core client for 30 sec - exiting 12:20:40 (6356): No heartbeat from core client for 30 sec - exiting 12:20:41 (6356): No heartbeat from core client for 30 sec - exiting 12:20:42 (6356): No heartbeat from core client for 30 sec - exiting 12:20:44 (6356): No heartbeat from core client for 30 sec - exiting 12:20:45 (6356): No heartbeat from core client for 30 sec - exiting 12:20:46 (6356): No heartbeat from core client for 30 sec - exiting 12:20:47 (6356): No heartbeat from core client for 30 sec - exiting 12:20:48 (6356): No heartbeat from core client for 30 sec - exiting 12:20:49 (6356): No heartbeat from core client for 30 sec - exiting 12:20:50 (6356): No heartbeat from core client for 30 sec - exiting 12:20:51 (6356): No heartbeat from core client for 30 sec - exiting 12:20:52 (6356): No heartbeat from core client for 30 sec - exiting 12:20:53 (6356): No heartbeat from core client for 30 sec - exiting 12:20:54 (6356): No heartbeat from core client for 30 sec - exiting 12:20:56 (6356): No heartbeat from core client for 30 sec - exiting 12:20:57 (6356): No heartbeat from core client for 30 sec - exiting 12:20:58 (6356): No heartbeat from core client for 30 sec - exiting 12:20:59 (6356): No heartbeat from core client for 30 sec - exiting 12:21:00 (6356): No heartbeat from core client for 30 sec - exiting 12:21:01 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:21:09 (7748): Can't acquire lockfile (32) - waiting 35s Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7748, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4808, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4808, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6812, iMonCtr=1 Model crash detected, will try to restart... 09:42:54 (6316): No heartbeat from core client for 30 sec - exiting 09:42:55 (6316): No heartbeat from core client for 30 sec - exiting 09:42:56 (6316): No heartbeat from core client for 30 sec - exiting 09:42:57 (6316): No heartbeat from core client for 30 sec - exiting 09:42:58 (6316): No heartbeat from core client for 30 sec - exiting 09:42:59 (6316): No heartbeat from core client for 30 sec - exiting 09:43:00 (6316): No heartbeat from core client for 30 sec - exiting 09:43:01 (6316): No heartbeat from core client for 30 sec - exiting 09:43:02 (6316): No heartbeat from core client for 30 sec - exiting 09:43:04 (6316): No heartbeat from core client for 30 sec - exiting 09:43:05 (6316): No heartbeat from core client for 30 sec - exiting 09:43:06 (6316): No heartbeat from core client for 30 sec - exiting 09:43:07 (6316): No heartbeat from core client for 30 sec - exiting 09:43:08 (6316): No heartbeat from core client for 30 sec - exiting 09:43:09 (6316): No heartbeat from core client for 30 sec - exiting 09:43:10 (6316): No heartbeat from core client for 30 sec - exiting 09:43:11 (6316): No heartbeat from core client for 30 sec - exiting 09:43:12 (6316): No heartbeat from core client for 30 sec - exiting 09:43:13 (6316): No heartbeat from core client for 30 sec - exiting 09:43:14 (6316): No heartbeat from core client for 30 sec - exiting 09:43:16 (6316): No heartbeat from core client for 30 sec - exiting 09:43:17 (6316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=972, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2244, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6252, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/85vqko.pjk0c10 Error converting file to netcdf: dataout/85vqko.pik0c10 Error converting file to netcdf: dataout/85vqko.pfk0c10 Error converting file to netcdf: dataout/85vqko.pck0c10 Error converting file to netcdf: dataout/85vqko.pbk0c10 Error converting file to netcdf: dataout/85vqko.pak0c10 Error converting file to netcdf: dataout/85vqka.phk0c10 Error converting file to netcdf: dataout/85vqka.pgk0c10 Error converting file to netcdf: dataout/85vqka.pek0c10 Error converting file to netcdf: dataout/85vqka.pdk0c10 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6504, iMonCtr=1 Model crash detected, will try to restart... 09:05:11 (7220): No heartbeat from core client for 30 sec - exiting 09:05:12 (7220): No heartbeat from core client for 30 sec - exiting 09:05:13 (7220): No heartbeat from core client for 30 sec - exiting 09:05:14 (7220): No heartbeat from core client for 30 sec - exiting 09:05:16 (7220): No heartbeat from core client for 30 sec - exiting 09:05:17 (7220): No heartbeat from core client for 30 sec - exiting 09:05:18 (7220): No heartbeat from core client for 30 sec - exiting 09:05:19 (7220): No heartbeat from core client for 30 sec - exiting 09:05:20 (7220): No heartbeat from core client for 30 sec - exiting 09:05:21 (7220): No heartbeat from core client for 30 sec - exiting 09:05:22 (7220): No heartbeat from core client for 30 sec - exiting 09:05:23 (7220): No heartbeat from core client for 30 sec - exiting 09:05:24 (7220): No heartbeat from core client for 30 sec - exiting 09:05:25 (7220): No heartbeat from core client for 30 sec - exiting 09:05:26 (7220): No heartbeat from core client for 30 sec - exiting 09:05:28 (7220): No heartbeat from core client for 30 sec - exiting 09:05:29 (7220): No heartbeat from core client for 30 sec - exiting 09:05:30 (7220): No heartbeat from core client for 30 sec - exiting 09:05:31 (7220): No heartbeat from core client for 30 sec - exiting 09:05:32 (7220): No heartbeat from core client for 30 sec - exiting 09:05:33 (7220): No heartbeat from core client for 30 sec - exiting 09:05:34 (7220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7548, iMonCtr=1 Model crash detected, will try to restart... 09:21:33 (3300): No heartbeat from core client for 30 sec - exiting 09:21:34 (3300): No heartbeat from core client for 30 sec - exiting 09:21:36 (3300): No heartbeat from core client for 30 sec - exiting 09:21:37 (3300): No heartbeat from core client for 30 sec - exiting 09:21:38 (3300): No heartbeat from core client for 30 sec - exiting 09:21:39 (3300): No heartbeat from core client for 30 sec - exiting 09:21:40 (3300): No heartbeat from core client for 30 sec - exiting 09:21:41 (3300): No heartbeat from core client for 30 sec - exiting 09:21:42 (3300): No heartbeat from core client for 30 sec - exiting 09:21:43 (3300): No heartbeat from core client for 30 sec - exiting 09:21:44 (3300): No heartbeat from core client for 30 sec - exiting 09:21:45 (3300): No heartbeat from core client for 30 sec - exiting 09:21:46 (3300): No heartbeat from core client for 30 sec - exiting 09:21:48 (3300): No heartbeat from core client for 30 sec - exiting 09:21:49 (3300): No heartbeat from core client for 30 sec - exiting 09:21:50 (3300): No heartbeat from core client for 30 sec - exiting 09:21:51 (3300): No heartbeat from core client for 30 sec - exiting 09:21:52 (3300): No heartbeat from core client for 30 sec - exiting 09:21:53 (3300): No heartbeat from core client for 30 sec - exiting 09:21:54 (3300): No heartbeat from core client for 30 sec - exiting 09:21:55 (3300): No heartbeat from core client for 30 sec - exiting 09:21:56 (3300): No heartbeat from core client for 30 sec - exiting 09:21:57 (3300): No heartbeat from core client for 30 sec - exiting 09:21:59 (3300): No heartbeat from core client for 30 sec - exiting 09:22:00 (3300): No heartbeat from core client for 30 sec - exiting 09:22:01 (3300): No heartbeat from core client for 30 sec - exiting 09:22:02 (3300): No heartbeat from core client for 30 sec - exiting 09:22:03 (3300): No heartbeat from core client for 30 sec - exiting 09:22:04 (3300): No heartbeat from core client for 30 sec - exiting 09:22:05 (3300): No heartbeat from core client for 30 sec - exiting 09:22:06 (3300): No heartbeat from core client for 30 sec - exiting 09:22:07 (3300): No heartbeat from core client for 30 sec - exiting 09:22:08 (3300): No heartbeat from core client for 30 sec - exiting 09:22:09 (3300): No heartbeat from core client for 30 sec - exiting 09:22:11 (3300): No heartbeat from core client for 30 sec - exiting 09:22:12 (3300): No heartbeat from core client for 30 sec - exiting 09:22:13 (3300): No heartbeat from core client for 30 sec - exiting 09:22:14 (3300): No heartbeat from core client for 30 sec - exiting 09:22:15 (3300): No heartbeat from core client for 30 sec - exiting 09:22:16 (3300): No heartbeat from core client for 30 sec - exiting 09:22:17 (3300): No heartbeat from core client for 30 sec - exiting 09:22:18 (3300): No heartbeat from core client for 30 sec - exiting 09:22:19 (3300): No heartbeat from core client for 30 sec - exiting 09:22:20 (3300): No heartbeat from core client for 30 sec - exiting 09:22:21 (3300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2720, iMonCtr=1 Model crash detected, will try to restart... 09:32:31 (4816): No heartbeat from core client for 30 sec - exiting 09:32:32 (4816): No heartbeat from core client for 30 sec - exiting 09:32:34 (4816): No heartbeat from core client for 30 sec - exiting 09:32:35 (4816): No heartbeat from core client for 30 sec - exiting 09:32:36 (4816): No heartbeat from core client for 30 sec - exiting 09:32:37 (4816): No heartbeat from core client for 30 sec - exiting 09:32:38 (4816): No heartbeat from core client for 30 sec - exiting 09:32:39 (4816): No heartbeat from core client for 30 sec - exiting 09:32:40 (4816): No heartbeat from core client for 30 sec - exiting 09:32:41 (4816): No heartbeat from core client for 30 sec - exiting 09:32:42 (4816): No heartbeat from core client for 30 sec - exiting 09:32:43 (4816): No heartbeat from core client for 30 sec - exiting 09:32:45 (4816): No heartbeat from core client for 30 sec - exiting 09:32:46 (4816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:32:47 (4816): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7348, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7348, iMonCtr=1 Model crash detected, will try to restart... 09:20:32 (8020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7416, iMonCtr=1 Model crash detected, will try to restart... 10:34:27 (6824): No heartbeat from core client for 30 sec - exiting 10:34:28 (6824): No heartbeat from core client for 30 sec - exiting 10:34:29 (6824): No heartbeat from core client for 30 sec - exiting 10:34:30 (6824): No heartbeat from core client for 30 sec - exiting 10:34:31 (6824): No heartbeat from core client for 30 sec - exiting 10:34:32 (6824): No heartbeat from core client for 30 sec - exiting 10:34:33 (6824): No heartbeat from core client for 30 sec - exiting 10:34:34 (6824): No heartbeat from core client for 30 sec - exiting 10:34:36 (6824): No heartbeat from core client for 30 sec - exiting 10:34:37 (6824): No heartbeat from core client for 30 sec - exiting 10:34:38 (6824): No heartbeat from core client for 30 sec - exiting 10:34:39 (6824): No heartbeat from core client for 30 sec - exiting 10:34:40 (6824): No heartbeat from core client for 30 sec - exiting 10:34:41 (6824): No heartbeat from core client for 30 sec - exiting 10:34:42 (6824): No heartbeat from core client for 30 sec - exiting 10:34:43 (6824): No heartbeat from core client for 30 sec - exiting 10:34:44 (6824): No heartbeat from core client for 30 sec - exiting 10:34:45 (6824): No heartbeat from core client for 30 sec - exiting 10:34:46 (6824): No heartbeat from core client for 30 sec - exiting 10:34:48 (6824): No heartbeat from core client for 30 sec - exiting 10:34:49 (6824): No heartbeat from core client for 30 sec - exiting 10:34:50 (6824): No heartbeat from core client for 30 sec - exiting 10:34:51 (6824): No heartbeat from core client for 30 sec - exiting 10:34:52 (6824): No heartbeat from core client for 30 sec - exiting 10:34:53 (6824): No heartbeat from core client for 30 sec - exiting 10:34:54 (6824): No heartbeat from core client for 30 sec - exiting 10:34:55 (6824): No heartbeat from core client for 30 sec - exiting 10:34:56 (6824): No heartbeat from core client for 30 sec - exiting 10:34:57 (6824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2848, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=224, iMonCtr=1 Model crash detected, will try to restart... 09:03:08 (7296): No heartbeat from core client for 30 sec - exiting 09:03:09 (7296): No heartbeat from core client for 30 sec - exiting 09:03:10 (7296): No heartbeat from core client for 30 sec - exiting 09:03:11 (7296): No heartbeat from core client for 30 sec - exiting 09:03:12 (7296): No heartbeat from core client for 30 sec - exiting 09:03:13 (7296): No heartbeat from core client for 30 sec - exiting 09:03:14 (7296): No heartbeat from core client for 30 sec - exiting 09:03:15 (7296): No heartbeat from core client for 30 sec - exiting 09:03:17 (7296): No heartbeat from core client for 30 sec - exiting 09:03:18 (7296): No heartbeat from core client for 30 sec - exiting 09:03:19 (7296): No heartbeat from core client for 30 sec - exiting 09:03:20 (7296): No heartbeat from core client for 30 sec - exiting 09:03:21 (7296): No heartbeat from core client for 30 sec - exiting 09:03:22 (7296): No heartbeat from core client for 30 sec - exiting 09:03:23 (7296): No heartbeat from core client for 30 sec - exiting 09:03:24 (7296): No heartbeat from core client for 30 sec - exiting 09:03:25 (7296): No heartbeat from core client for 30 sec - exiting 09:03:26 (7296): No heartbeat from core client for 30 sec - exiting 09:03:27 (7296): No heartbeat from core client for 30 sec - exiting 09:03:29 (7296): No heartbeat from core client for 30 sec - exiting 09:03:30 (7296): No heartbeat from core client for 30 sec - exiting 09:03:31 (7296): No heartbeat from core client for 30 sec - exiting 09:03:32 (7296): No heartbeat from core client for 30 sec - exiting 09:03:33 (7296): No heartbeat from core client for 30 sec - exiting 09:03:34 (7296): No heartbeat from core client for 30 sec - exiting 09:03:35 (7296): No heartbeat from core client for 30 sec - exiting 09:03:36 (7296): No heartbeat from core client for 30 sec - exiting 09:03:37 (7296): No heartbeat from core client for 30 sec - exiting 09:03:38 (7296): No heartbeat from core client for 30 sec - exiting 09:03:39 (7296): No heartbeat from core client for 30 sec - exiting 09:03:41 (7296): No heartbeat from core client for 30 sec - exiting 09:03:42 (7296): No heartbeat from core client for 30 sec - exiting 09:03:43 (7296): No heartbeat from core client for 30 sec - exiting 09:03:44 (7296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3932, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2464, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3208, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1252, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 May 2014 11:08:34	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	803,520	1,515,229	1.8857
11 May 2014 20:13:03	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	777,600	1,467,137	1.8868
09 May 2014 13:38:01	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	751,680	1,418,229	1.8867
07 May 2014 17:23:02	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	725,760	1,368,958	1.8862
06 May 2014 16:09:02	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	699,840	1,319,450	1.8854
04 May 2014 16:03:25	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	673,920	1,270,400	1.8851
02 May 2014 20:29:52	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	648,000	1,221,538	1.8851
01 May 2014 10:22:51	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	622,080	1,172,444	1.8847
29 Apr 2014 20:39:05	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	596,160	1,124,036	1.8855
28 Apr 2014 16:41:00	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	570,240	1,075,008	1.8852
28 Apr 2014 02:17:44	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	544,320	1,026,213	1.8853
27 Apr 2014 11:54:20	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	518,400	977,551	1.8857
24 Apr 2014 10:05:11	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	492,480	928,799	1.8860
23 Apr 2014 09:15:50	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	466,560	879,392	1.8848
21 Apr 2014 20:58:17	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	440,640	830,148	1.8840
19 Apr 2014 19:29:39	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	414,720	781,483	1.8844
16 Apr 2014 12:16:15	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	388,800	733,065	1.8855
15 Apr 2014 08:54:00	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	362,880	683,836	1.8845
13 Apr 2014 18:17:26	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	336,960	634,820	1.8840
11 Apr 2014 11:28:41	1143449	16389942	hadcm3n_85vq_1980_40_008514125_2	311,040	585,774	1.8833