Task 12821405

Name	hadcm3n_p2fp_1900_40_007220741_1
Workunit	7418981
Created	26 Apr 2011, 15:21:01 UTC
Sent	2 May 2011, 13:35:40 UTC
Report deadline	1 Aug 2011, 21:02:51 UTC
Received	22 Jul 2011, 13:07:15 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1027037
Run time	32 days 6 hours 31 min 26 sec
CPU time	27 days 21 hours 54 min 4 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.08 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6600, iMonCtr=1 Model crash detected, will try to restart... 08:21:48 (4412): No heartbeat from core client for 30 sec - exiting 08:21:51 (4412): No heartbeat from core client for 30 sec - exiting 08:21:52 (4412): No heartbeat from core client for 30 sec - exiting 08:21:53 (4412): No heartbeat from core client for 30 sec - exiting 08:21:54 (4412): No heartbeat from core client for 30 sec - exiting 08:21:55 (4412): No heartbeat from core client for 30 sec - exiting 08:21:56 (4412): No heartbeat from core client for 30 sec - exiting 08:21:57 (4412): No heartbeat from core client for 30 sec - exiting 08:21:58 (4412): No heartbeat from core client for 30 sec - exiting 08:21:59 (4412): No heartbeat from core client for 30 sec - exiting 08:22:00 (4412): No heartbeat from core client for 30 sec - exiting 08:22:01 (4412): No heartbeat from core client for 30 sec - exiting 08:22:02 (4412): No heartbeat from core client for 30 sec - exiting 08:22:03 (4412): No heartbeat from core client for 30 sec - exiting 08:22:04 (4412): No heartbeat from core client for 30 sec - exiting 08:22:05 (4412): No heartbeat from core client for 30 sec - exiting 08:22:06 (4412): No heartbeat from core client for 30 sec - exiting 08:22:07 (4412): No heartbeat from core client for 30 sec - exiting 08:22:08 (4412): No heartbeat from core client for 30 sec - exiting 08:22:09 (4412): No heartbeat from core client for 30 sec - exiting 08:22:10 (4412): No heartbeat from core client for 30 sec - exiting 08:22:11 (4412): No heartbeat from core client for 30 sec - exiting 08:22:12 (4412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5780, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6132, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 14:05:14 (5948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5836, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5948, iMonCtr=1 Model crash detected, will try to restart... 08:12:56 (5868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:59:37 (5436): No heartbeat from core client for 30 sec - exiting 10:59:38 (5436): No heartbeat from core client for 30 sec - exiting 10:59:39 (5436): No heartbeat from core client for 30 sec - exiting 10:59:40 (5436): No heartbeat from core client for 30 sec - exiting 10:59:41 (5436): No heartbeat from core client for 30 sec - exiting 10:59:42 (5436): No heartbeat from core client for 30 sec - exiting 10:59:43 (5436): No heartbeat from core client for 30 sec - exiting 10:59:44 (5436): No heartbeat from core client for 30 sec - exiting 10:59:45 (5436): No heartbeat from core client for 30 sec - exiting 10:59:46 (5436): No heartbeat from core client for 30 sec - exiting 10:59:47 (5436): No heartbeat from core client for 30 sec - exiting 10:59:48 (5436): No heartbeat from core client for 30 sec - exiting 10:59:49 (5436): No heartbeat from core client for 30 sec - exiting 10:59:50 (5436): No heartbeat from core client for 30 sec - exiting 10:59:51 (5436): No heartbeat from core client for 30 sec - exiting 10:59:52 (5436): No heartbeat from core client for 30 sec - exiting 10:59:53 (5436): No heartbeat from core client for 30 sec - exiting 10:59:54 (5436): No heartbeat from core client for 30 sec - exiting 10:59:55 (5436): No heartbeat from core client for 30 sec - exiting 10:59:56 (5436): No heartbeat from core client for 30 sec - exiting 10:59:57 (5436): No heartbeat from core client for 30 sec - exiting 10:59:58 (5436): No heartbeat from core client for 30 sec - exiting 10:59:59 (5436): No heartbeat from core client for 30 sec - exiting 11:00:00 (5436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5244, iMonCtr=1 Model crash detected, will try to restart... 14:20:39 (4612): No heartbeat from core client for 30 sec - exiting 14:20:40 (4612): No heartbeat from core client for 30 sec - exiting 14:20:41 (4612): No heartbeat from core client for 30 sec - exiting 14:20:42 (4612): No heartbeat from core client for 30 sec - exiting 14:20:43 (4612): No heartbeat from core client for 30 sec - exiting 14:20:44 (4612): No heartbeat from core client for 30 sec - exiting 14:20:45 (4612): No heartbeat from core client for 30 sec - exiting 14:20:46 (4612): No heartbeat from core client for 30 sec - exiting 14:20:47 (4612): No heartbeat from core client for 30 sec - exiting 14:20:48 (4612): No heartbeat from core client for 30 sec - exiting 14:20:49 (4612): No heartbeat from core client for 30 sec - exiting 14:20:50 (4612): No heartbeat from core client for 30 sec - exiting 14:20:51 (4612): No heartbeat from core client for 30 sec - exiting 14:20:52 (4612): No heartbeat from core client for 30 sec - exiting 14:20:53 (4612): No heartbeat from core client for 30 sec - exiting 14:20:54 (4612): No heartbeat from core client for 30 sec - exiting 14:20:55 (4612): No heartbeat from core client for 30 sec - exiting 14:20:56 (4612): No heartbeat from core client for 30 sec - exiting 14:20:57 (4612): No heartbeat from core client for 30 sec - exiting 14:20:58 (4612): No heartbeat from core client for 30 sec - exiting 14:20:59 (4612): No heartbeat from core client for 30 sec - exiting 14:21:00 (4612): No heartbeat from core client for 30 sec - exiting 14:21:01 (4612): No heartbeat from core client for 30 sec - exiting 14:21:02 (4612): No heartbeat from core client for 30 sec - exiting 14:21:03 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:24:08 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5184, iMonCtr=1 Model crash detected, will try to restart... 18:00:09 (5208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:04:23 (584): No heartbeat from core client for 30 sec - exiting 08:04:24 (584): No heartbeat from core client for 30 sec - exiting 08:04:25 (584): No heartbeat from core client for 30 sec - exiting 08:04:27 (584): No heartbeat from core client for 30 sec - exiting 08:04:28 (584): No heartbeat from core client for 30 sec - exiting 08:04:29 (584): No heartbeat from core client for 30 sec - exiting 08:04:30 (584): No heartbeat from core client for 30 sec - exiting 08:04:31 (584): No heartbeat from core client for 30 sec - exiting 08:04:32 (584): No heartbeat from core client for 30 sec - exiting 08:04:33 (584): No heartbeat from core client for 30 sec - exiting 08:04:34 (584): No heartbeat from core client for 30 sec - exiting 08:04:35 (584): No heartbeat from core client for 30 sec - exiting 08:04:36 (584): No heartbeat from core client for 30 sec - exiting 08:04:37 (584): No heartbeat from core client for 30 sec - exiting 08:04:38 (584): No heartbeat from core client for 30 sec - exiting 08:04:39 (584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:06:27 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:47 (3420): No heartbeat from core client for 30 sec - exiting 10:24:48 (3420): No heartbeat from core client for 30 sec - exiting 10:24:49 (3420): No heartbeat from core client for 30 sec - exiting 10:24:50 (3420): No heartbeat from core client for 30 sec - exiting 10:24:51 (3420): No heartbeat from core client for 30 sec - exiting 10:24:52 (3420): No heartbeat from core client for 30 sec - exiting 10:24:53 (3420): No heartbeat from core client for 30 sec - exiting 10:24:54 (3420): No heartbeat from core client for 30 sec - exiting 10:24:55 (3420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:28:07 (640): No heartbeat from core client for 30 sec - exiting 07:28:08 (640): No heartbeat from core client for 30 sec - exiting 07:28:09 (640): No heartbeat from core client for 30 sec - exiting 07:28:10 (640): No heartbeat from core client for 30 sec - exiting 07:28:11 (640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:07:54 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:07:55 (4456): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4716, iMonCtr=1 Model crash detected, will try to restart... 06:53:41 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4964, iMonCtr=1 Model crash detected, will try to restart... 08:05:45 (2412): No heartbeat from core client for 30 sec - exiting 08:05:46 (2412): No heartbeat from core client for 30 sec - exiting 08:05:47 (2412): No heartbeat from core client for 30 sec - exiting 08:05:48 (2412): No heartbeat from core client for 30 sec - exiting 08:05:49 (2412): No heartbeat from core client for 30 sec - exiting 08:05:50 (2412): No heartbeat from core client for 30 sec - exiting 08:05:51 (2412): No heartbeat from core client for 30 sec - exiting 08:05:52 (2412): No heartbeat from core client for 30 sec - exiting 08:05:53 (2412): No heartbeat from core client for 30 sec - exiting 08:05:54 (2412): No heartbeat from core client for 30 sec - exiting 08:05:55 (2412): No heartbeat from core client for 30 sec - exiting 08:05:56 (2412): No heartbeat from core client for 30 sec - exiting 08:05:57 (2412): No heartbeat from core client for 30 sec - exiting 08:05:58 (2412): No heartbeat from core client for 30 sec - exiting 08:05:59 (2412): No heartbeat from core client for 30 sec - exiting 08:06:00 (2412): No heartbeat from core client for 30 sec - exiting 08:06:01 (2412): No heartbeat from core client for 30 sec - exiting 08:06:02 (2412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2696, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5388, iMonCtr=1 Model crash detected, will try to restart... 12:45:58 (4528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:45:59 (4528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:23:05 (4400): No heartbeat from core client for 30 sec - exiting 13:23:07 (4400): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:23:08 (4400): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:24:22 (2908): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 15:24:23 (2908): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:15:21 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:23 (5512): No heartbeat from core client for 30 sec - exiting 07:15:24 (5512): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 07:30:51 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:30:52 (5116): No heartbeat from core client for 30 sec - exiting 07:31:33 (576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:21:09 (5824): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 12:21:11 (5824): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5616, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:55:55 (4776): No heartbeat from core client for 30 sec - exiting 10:55:56 (4776): No heartbeat from core client for 30 sec - exiting 10:55:57 (4776): No heartbeat from core client for 30 sec - exiting 10:55:58 (4776): No heartbeat from core client for 30 sec - exiting 10:55:59 (4776): No heartbeat from core client for 30 sec - exiting 10:56:00 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:12:58 (5056): No heartbeat from core client for 30 sec - exiting 09:12:59 (5056): No heartbeat from core client for 30 sec - exiting 09:13:00 (5056): No heartbeat from core client for 30 sec - exiting 09:13:01 (5056): No heartbeat from core client for 30 sec - exiting 09:13:02 (5056): No heartbeat from core client for 30 sec - exiting 09:13:03 (5056): No heartbeat from core client for 30 sec - exiting 09:13:04 (5056): No heartbeat from core client for 30 sec - exiting 09:13:05 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:06 (5056): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:58:37 (5416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:39:06 (5652): No heartbeat from core client for 30 sec - exiting 08:39:07 (5652): No heartbeat from core client for 30 sec - exiting 08:39:08 (5652): No heartbeat from core client for 30 sec - exiting 08:39:09 (5652): No heartbeat from core client for 30 sec - exiting 08:39:10 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:40:36 (4452): No heartbeat from core client for 30 sec - exiting 08:40:37 (4452): No heartbeat from core client for 30 sec - exiting 08:40:38 (4452): No heartbeat from core client for 30 sec - exiting 08:40:39 (4452): No heartbeat from core client for 30 sec - exiting 08:40:40 (4452): No heartbeat from core client for 30 sec - exiting 08:40:41 (4452): No heartbeat from core client for 30 sec - exiting 08:40:42 (4452): No heartbeat from core client for 30 sec - exiting 08:40:43 (4452): No heartbeat from core client for 30 sec - exiting 08:40:44 (4452): No heartbeat from core client for 30 sec - exiting 08:40:45 (4452): No heartbeat from core client for 30 sec - exiting 08:40:46 (4452): No heartbeat from core client for 30 sec - exiting 08:40:47 (4452): No heartbeat from core client for 30 sec - exiting 08:40:48 (4452): No heartbeat from core client for 30 sec - exiting 08:40:49 (4452): No heartbeat from core client for 30 sec - exiting 08:40:50 (4452): No heartbeat from core client for 30 sec - exiting 08:40:51 (4452): No heartbeat from core client for 30 sec - exiting 08:40:52 (4452): No heartbeat from core client for 30 sec - exiting 08:40:53 (4452): No heartbeat from core client for 30 sec - exiting 08:40:54 (4452): No heartbeat from core client for 30 sec - exiting 08:40:55 (4452): No heartbeat from core client for 30 sec - exiting 08:40:56 (4452): No heartbeat from core client for 30 sec - exiting 08:40:57 (4452): No heartbeat from core client for 30 sec - exiting 08:40:58 (4452): No heartbeat from core client for 30 sec - exiting 08:40:59 (4452): No heartbeat from core client for 30 sec - exiting 08:41:00 (4452): No heartbeat from core client for 30 sec - exiting 08:41:01 (4452): No heartbeat from core client for 30 sec - exiting 08:41:02 (4452): No heartbeat from core client for 30 sec - exiting 08:41:03 (4452): No heartbeat from core client for 30 sec - exiting 08:41:04 (4452): No heartbeat from core client for 30 sec - exiting 08:41:05 (4452): No heartbeat from core client for 30 sec - exiting 08:41:06 (4452): No heartbeat from core client for 30 sec - exiting 08:41:07 (4452): No heartbeat from core client for 30 sec - exiting 08:41:08 (4452): No heartbeat from core client for 30 sec - exiting 08:41:09 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:41:10 (4452): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 09:56:06 (4792): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 09:56:07 (4792): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1916, iMonCtr=1 Model crash detected, will try to restart... 17:26:31 (4160): No heartbeat from core client for 30 sec - exiting 17:26:32 (4160): No heartbeat from core client for 30 sec - exiting 17:26:33 (4160): No heartbeat from core client for 30 sec - exiting 17:26:34 (4160): No heartbeat from core client for 30 sec - exiting 17:26:35 (4160): No heartbeat from core client for 30 sec - exiting 17:26:36 (4160): No heartbeat from core client for 30 sec - exiting 17:26:37 (4160): No heartbeat from core client for 30 sec - exiting 17:26:38 (4160): No heartbeat from core client for 30 sec - exiting 17:26:39 (4160): No heartbeat from core client for 30 sec - exiting 17:26:40 (4160): No heartbeat from core client for 30 sec - exiting 17:26:41 (4160): No heartbeat from core client for 30 sec - exiting 17:26:42 (4160): No heartbeat from core client for 30 sec - exiting 17:26:43 (4160): No heartbeat from core client for 30 sec - exiting 17:26:44 (4160): No heartbeat from core client for 30 sec - exiting 17:26:45 (4160): No heartbeat from core client for 30 sec - exiting 17:26:46 (4160): No heartbeat from core client for 30 sec - exiting 17:26:47 (4160): No heartbeat from core client for 30 sec - exiting 17:26:48 (4160): No heartbeat from core client for 30 sec - exiting 17:26:49 (4160): No heartbeat from core client for 30 sec - exiting 17:26:50 (4160): No heartbeat from core client for 30 sec - exiting 17:26:51 (4160): No heartbeat from core client for 30 sec - exiting 17:26:52 (4160): No heartbeat from core client for 30 sec - exiting 17:26:53 (4160): No heartbeat from core client for 30 sec - exiting 17:26:54 (4160): No heartbeat from core client for 30 sec - exiting 17:26:55 (4160): No heartbeat from core client for 30 sec - exiting 17:26:56 (4160): No heartbeat from core client for 30 sec - exiting 17:26:57 (4160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4444, iMonCtr=1 Model crash detected, will try to restart... 08:35:06 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:20:17 (5976): No heartbeat from core client for 30 sec - exiting 13:20:18 (5976): No heartbeat from core client for 30 sec - exiting 13:20:19 (5976): No heartbeat from core client for 30 sec - exiting 13:20:20 (5976): No heartbeat from core client for 30 sec - exiting 13:20:21 (5976): No heartbeat from core client for 30 sec - exiting 13:20:22 (5976): No heartbeat from core client for 30 sec - exiting 13:20:23 (5976): No heartbeat from core client for 30 sec - exiting 13:20:24 (5976): No heartbeat from core client for 30 sec - exiting 13:20:25 (5976): No heartbeat from core client for 30 sec - exiting 13:20:26 (5976): No heartbeat from core client for 30 sec - exiting 13:20:27 (5976): No heartbeat from core client for 30 sec - exiting 13:20:28 (5976): No heartbeat from core client for 30 sec - exiting 13:20:29 (5976): No heartbeat from core client for 30 sec - exiting 13:20:30 (5976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:31 (5976): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:51:02 (1956): No heartbeat from core client for 30 sec - exiting 08:51:03 (1956): No heartbeat from core client for 30 sec - exiting 08:51:04 (1956): No heartbeat from core client for 30 sec - exiting 08:51:05 (1956): No heartbeat from core client for 30 sec - exiting 08:51:06 (1956): No heartbeat from core client for 30 sec - exiting 08:51:07 (1956): No heartbeat from core client for 30 sec - exiting 08:51:08 (1956): No heartbeat from core client for 30 sec - exiting 08:51:09 (1956): No heartbeat from core client for 30 sec - exiting 08:51:10 (1956): No heartbeat from core client for 30 sec - exiting 08:51:11 (1956): No heartbeat from core client for 30 sec - exiting 08:51:12 (1956): No heartbeat from core client for 30 sec - exiting 08:51:13 (1956): No heartbeat from core client for 30 sec - exiting 08:51:14 (1956): No heartbeat from core client for 30 sec - exiting 08:51:15 (1956): No heartbeat from core client for 30 sec - exiting 08:51:16 (1956): No heartbeat from core client for 30 sec - exiting 08:51:17 (1956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5732, iMonCtr=1 Model crash detected, will try to restart... 09:23:21 (5236): No heartbeat from core client for 30 sec - exiting 09:23:22 (5236): No heartbeat from core client for 30 sec - exiting 09:23:23 (5236): No heartbeat from core client for 30 sec - exiting 09:23:24 (5236): No heartbeat from core client for 30 sec - exiting 09:23:25 (5236): No heartbeat from core client for 30 sec - exiting 09:23:26 (5236): No heartbeat from core client for 30 sec - exiting 09:23:27 (5236): No heartbeat from core client for 30 sec - exiting 09:23:28 (5236): No heartbeat from core client for 30 sec - exiting 09:23:29 (5236): No heartbeat from core client for 30 sec - exiting 09:23:30 (5236): No heartbeat from core client for 30 sec - exiting 09:23:31 (5236): No heartbeat from core client for 30 sec - exiting 09:23:32 (5236): No heartbeat from core client for 30 sec - exiting 09:23:33 (5236): No heartbeat from core client for 30 sec - exiting 09:23:34 (5236): No heartbeat from core client for 30 sec - exiting 09:23:35 (5236): No heartbeat from core client for 30 sec - exiting 09:23:36 (5236): No heartbeat from core client for 30 sec - exiting 09:23:37 (5236): No heartbeat from core client for 30 sec - exiting 09:23:38 (5236): No heartbeat from core client for 30 sec - exiting 09:23:39 (5236): No heartbeat from core client for 30 sec - exiting 09:23:40 (5236): No heartbeat from core client for 30 sec - exiting 09:23:41 (5236): No heartbeat from core client for 30 sec - exiting 09:23:42 (5236): No heartbeat from core client for 30 sec - exiting 09:23:43 (5236): No heartbeat from core client for 30 sec - exiting 09:23:44 (5236): No heartbeat from core client for 30 sec - exiting 09:23:45 (5236): No heartbeat from core client for 30 sec - exiting 09:23:46 (5236): No heartbeat from core client for 30 sec - exiting 09:23:47 (5236): No heartbeat from core client for 30 sec - exiting 09:23:48 (5236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:17:26 (4008): No heartbeat from core client for 30 sec - exiting 15:17:27 (4008): No heartbeat from core client for 30 sec - exiting 15:17:28 (4008): No heartbeat from core client for 30 sec - exiting 15:17:29 (4008): No heartbeat from core client for 30 sec - exiting 15:17:30 (4008): No heartbeat from core client for 30 sec - exiting 15:17:31 (4008): No heartbeat from core client for 30 sec - exiting 15:17:32 (4008): No heartbeat from core client for 30 sec - exiting 15:17:33 (4008): No heartbeat from core client for 30 sec - exiting 15:17:34 (4008): No heartbeat from core client for 30 sec - exiting 15:17:35 (4008): No heartbeat from core client for 30 sec - exiting 15:17:36 (4008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:17:37 (4008): No heartbeat from core client for 30 sec - exiting 15:17:38 (4008): No heartbeat from core client for 30 sec - exiting 15:17:39 (4008): No heartbeat from core client for 30 sec - exiting 15:17:40 (4008): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:24:40 (5244): No heartbeat from core client for 30 sec - exiting 07:24:41 (5244): No heartbeat from core client for 30 sec - exiting 07:24:42 (5244): No heartbeat from core client for 30 sec - exiting 07:24:43 (5244): No heartbeat from core client for 30 sec - exiting 07:24:44 (5244): No heartbeat from core client for 30 sec - exiting 07:24:45 (5244): No heartbeat from core client for 30 sec - exiting 07:24:46 (5244): No heartbeat from core client for 30 sec - exiting 07:24:47 (5244): No heartbeat from core client for 30 sec - exiting 07:24:48 (5244): No heartbeat from core client for 30 sec - exiting 07:24:49 (5244): No heartbeat from core client for 30 sec - exiting 07:24:50 (5244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:47:27 (5636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:43:02 (5340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:33:54 (5900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:33:55 (5900): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:32:42 (6112): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:32:43 (6112): No heartbeat from core client for 30 sec - exiting 13:50:15 (4524): No heartbeat from core client for 30 sec - exiting 13:50:16 (4524): No heartbeat from core client for 30 sec - exiting 13:50:17 (4524): No heartbeat from core client for 30 sec - exiting 13:50:18 (4524): No heartbeat from core client for 30 sec - exiting 13:50:19 (4524): No heartbeat from core client for 30 sec - exiting 13:50:20 (4524): No heartbeat from core client for 30 sec - exiting 13:50:21 (4524): No heartbeat from core client for 30 sec - exiting 13:50:22 (4524): No heartbeat from core client for 30 sec - exiting 13:50:23 (4524): No heartbeat from core client for 30 sec - exiting 13:50:24 (4524): No heartbeat from core client for 30 sec - exiting 13:50:25 (4524): No heartbeat from core client for 30 sec - exiting 13:50:26 (4524): No heartbeat from core client for 30 sec - exiting 13:50:27 (4524): No heartbeat from core client for 30 sec - exiting 13:50:28 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:07:07 (4184): No heartbeat from core client for 30 sec - exiting 11:07:08 (4184): No heartbeat from core client for 30 sec - exiting 11:07:09 (4184): No heartbeat from core client for 30 sec - exiting 11:07:10 (4184): No heartbeat from core client for 30 sec - exiting 11:07:11 (4184): No heartbeat from core client for 30 sec - exiting 11:07:12 (4184): No heartbeat from core client for 30 sec - exiting 11:07:13 (4184): No heartbeat from core client for 30 sec - exiting 11:07:14 (4184): No heartbeat from core client for 30 sec - exiting 11:07:15 (4184): No heartbeat from core client for 30 sec - exiting 11:07:16 (4184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:08:15 (6032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5936, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 21:59:29 (1540): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 15:23:51 (4248): No heartbeat from core client for 30 sec - exiting 15:23:52 (4248): No heartbeat from core client for 30 sec - exiting 15:23:53 (4248): No heartbeat from core client for 30 sec - exiting 15:23:54 (4248): No heartbeat from core client for 30 sec - exiting 15:23:55 (4248): No heartbeat from core client for 30 sec - exiting 15:23:56 (4248): No heartbeat from core client for 30 sec - exiting 15:23:57 (4248): No heartbeat from core client for 30 sec - exiting 15:23:58 (4248): No heartbeat from core client for 30 sec - exiting 15:23:59 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:32:52 (5720): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5132, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 17:47:06	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	1,036,800	2,410,964	2.3254
25 Jul 2011 15:49:13	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	1,010,880	2,350,894	2.3256
10 Jul 2011 01:18:53	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	984,960	2,287,634	2.3226
08 Jul 2011 21:09:52	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	959,040	2,227,279	2.3224
07 Jul 2011 15:48:58	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	933,120	2,168,715	2.3242
04 Jul 2011 17:15:40	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	907,200	2,110,777	2.3267
02 Jul 2011 17:49:25	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	881,280	2,052,607	2.3291
30 Jun 2011 20:22:57	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	855,360	1,994,943	2.3323
28 Jun 2011 19:46:07	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	829,440	1,935,688	2.3337
26 Jun 2011 23:01:03	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	803,520	1,877,455	2.3365
24 Jun 2011 19:31:13	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	777,600	1,820,666	2.3414
22 Jun 2011 21:17:49	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	751,680	1,761,999	2.3441
20 Jun 2011 22:38:26	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	725,760	1,703,632	2.3474
19 Jun 2011 23:10:20	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	699,840	1,645,444	2.3512
17 Jun 2011 01:15:47	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	673,920	1,586,047	2.3535
14 Jun 2011 22:19:34	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	648,000	1,526,836	2.3562
13 Jun 2011 00:41:32	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	622,080	1,468,251	2.3602
10 Jun 2011 12:15:27	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	596,160	1,408,200	2.3621
05 Jun 2011 15:12:58	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	570,240	1,351,138	2.3694
03 Jun 2011 12:19:32	1027037	12821405	hadcm3n_p2fp_1900_40_007220741_1	544,320	1,289,847	2.3696