Task 12791047

Name	hadam3p_pnw_z7z1_1972_1_006940853_2
Workunit	7144169
Created	10 Apr 2011, 6:43:15 UTC
Sent	10 Apr 2011, 8:57:56 UTC
Report deadline	22 Mar 2012, 14:17:56 UTC
Received	28 May 2011, 21:10:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	939489
Run time	3 days 19 hours 11 min 30 sec
CPU time	2 days 23 hours 6 min 57 sec
Validate state	Invalid
Credit	1,003.35
Device peak FLOPS	2.17 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2612, selfPID=2612, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN16:32:03 (3596): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3908, selfPID=3908, iMonCtr=2 21:30:17 (254108): srt_timer_thrhread(): CreateThread() failed, rrrno 0 no 0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2640, selfPID=5084, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3176, selfPID=3176, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4780, selfPID=4780, iMonCtr=2 11:08:21 (4512): start_timer_thread(): CreateThread() failed, errno 0 11:08:21 (4780): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 16:25:35 (2632): No heartbeat from core client for 30 sec - exiting 16:25:37 (2632): No heartbeat from core client for 30 sec - exiting 16:25:38 (2632): No heartbeat from core client for 30 sec - exiting 16:25:39 (2632): No heartbeat from core client for 30 sec - exiting 16:25:40 (2632): No heartbeat from core client for 30 sec - exiting 16:25:41 (2632): No heartbeat from core client for 30 sec - exiting 16:25:42 (2632): No heartbeat from core client for 30 sec - exiting 16:25:43 (2632): No heartbeat from core client for 30 sec - exiting 16:25:44 (2632): No heartbeat from core client for 30 sec - exiting 16:25:45 (2632): No heartbeat from core client for 30 sec - exiting 16:25:46 (2632): No heartbeat from core client for 30 sec - exiting 16:25:47 (2632): No heartbeat from core client for 30 sec - exiting 16:25:48 (2632): No heartbeat from core client for 30 sec - exiting 16:25:49 (2632): No heartbeat from core client for 30 sec - exiting 16:25:50 (2632): No heartbeat from core client for 30 sec - exiting 16:25:51 (2632): No heartbeat from core client for 30 sec - exiting 16:25:53 (2632): No heartbeat from core client for 30 sec - exiting 16:25:54 (2632): No heartbeat from core client for 30 sec - exiting 16:25:55 (2632): No heartbeat from core client for 30 sec - exiting 16:25:56 (2632): No heartbeat from core client for 30 sec - exiting 16:25:57 (2632): No heartbeat from core client for 30 sec - exiting 16:25:58 (2632): No heartbeat from core client for 30 sec - exiting 16:25:59 (2632): No heartbeat from core client for 30 sec - exiting 16:26:00 (2632): No heartbeat from core client for 30 sec - exiting 16:26:01 (2632): No heartbeat from core client for 30 sec - exiting 16:26:02 (2632): No heartbeat from core client for 30 sec - exiting 16:26:04 (2632): No heartbeat from core client for 30 sec - exiting 16:26:05 (2632): No heartbeat from core client for 30 sec - exiting 16:26:06 (2632): No heartbeat from core client for 30 sec - exiting 16:26:07 (2632): No heartbeat from core client for 30 sec - exiting 16:26:08 (2632): No heartbeat from core client for 30 sec - exiting 16:26:09 (2632): No heartbeat from core client for 30 sec - exiting 16:26:10 (2632): No heartbeat from core client for 30 sec - exiting 16:26:11 (2632): No heartbeat from core client for 30 sec - exiting 16:26:12 (2632): No heartbeat from core client for 30 sec - exiting 16:26:13 (2632): No heartbeat from core client for 30 sec - exiting 16:26:14 (2632): No heartbeat from core client for 30 sec - exiting 16:26:16 (2632): No heartbeat from core client for 30 sec - exiting 16:26:17 (2632): No heartbeat from core client for 30 sec - exiting 16:26:18 (2632): No heartbeat from core client for 30 sec - exiting 16:26:20 (2632): No heartbeat from core client for 30 sec - exiting 16:26:21 (2632): No heartbeat from core client for 30 sec - exiting 16:26:22 (2632): No heartbeat from core client for 30 sec - exiting 16:26:23 (2632): No heartbeat from core client for 30 sec - exiting 16:26:24 (2632): No heartbeat from core client for 30 sec - exiting 16:26:25 (2632): No heartbeat from core client for 30 sec - exiting 16:26:26 (2632): No heartbeat from core client for 30 sec - exiting 16:26:27 (2632): No heartbeat from core client for 30 sec - exiting 16:26:29 (2632): No heartbeat from core client for 30 sec - exiting 16:26:30 (2632): No heartbeat from core client for 30 sec - exiting 16:26:31 (2632): No heartbeat from core client for 30 sec - exiting 16:26:32 (2632): No heartbeat from core client for 30 sec - exiting 16:26:33 (2632): No heartbeat from core client for 30 sec - exiting 16:26:34 (2632): No heartbeat from core client for 30 sec - exiting 16:26:35 (2632): No heartbeat from core client for 30 sec - exiting 16:26:36 (2632): No heartbeat from core client for 30 sec - exiting 16:26:37 (2632): No heartbeat from core client for 30 sec - exiting 16:26:38 (2632): No heartbeat from core client for 30 sec - exiting 16:26:40 (2632): No heartbeat from core client for 30 sec - exiting 16:26:41 (2632): No heartbeat from core client for 30 sec - exiting 16:26:42 (2632): No heartbeat from core client for 30 sec - exiting 16:26:43 (2632): No heartbeat from core client for 30 sec - exiting 16:26:44 (2632): No heartbeat from core client for 30 sec - exiting 16:26:45 (2632): No heartbeat from core client for 30 sec - exiting 16:26:46 (2632): No heartbeat from core client for 30 sec - exiting 16:26:47 (2632): No heartbeat from core client for 30 sec - exiting 16:26:48 (2632): No heartbeat from core client for 30 sec - exiting 16:26:50 (2632): No heartbeat from core client for 30 sec - exiting 16:26:51 (2632): No heartbeat from core client for 30 sec - exiting 16:26:52 (2632): No heartbeat from core client for 30 sec - exiting 16:26:53 (2632): No heartbeat from core client for 30 sec - exiting 16:26:54 (2632): No heartbeat from core client for 30 sec - exiting 16:26:55 (2632): No heartbeat from core client for 30 sec - exiting 16:26:56 (2632): No heartbeat from core client for 30 sec - exiting 16:26:57 (2632): No heartbeat from core client for 30 sec - exiting 16:26:58 (2632): No heartbeat from core client for 30 sec - exiting 16:26:59 (2632): No heartbeat from core client for 30 sec - exiting 16:27:00 (2632): No heartbeat from core client for 30 sec - exiting 16:27:01 (2632): No heartbeat from core client for 30 sec - exiting 16:27:02 (2632): No heartbeat from core client for 30 sec - exiting 16:27:03 (2632): No heartbeat from core client for 30 sec - exiting 16:27:04 (2632): No heartbeat from core client for 30 sec - exiting 16:27:06 (2632): No heartbeat from core client for 30 sec - exiting 16:27:07 (2632): No heartbeat from core client for 30 sec - exiting 16:27:08 (2632): No heartbeat from core client for 30 sec - exiting 16:27:09 (2632): No heartbeat from core client for 30 sec - exiting 16:27:10 (2632): No heartbeat from core client for 30 sec - exiting 16:27:11 (2632): No heartbeat from core client for 30 sec - exiting 16:27:12 (2632): No heartbeat from core client for 30 sec - exiting 16:27:13 (2632): No heartbeat from core client for 30 sec - exiting 16:27:14 (2632): No heartbeat from core client for 30 sec - exiting 16:27:15 (2632): No heartbeat from core client for 30 sec - exiting 16:27:16 (2632): No heartbeat from core client for 30 sec - exiting 16:27:17 (2632): No heartbeat from core client for 30 sec - exiting 16:27:18 (2632): No heartbeat from core client for 30 sec - exiting 16:27:19 (2632): No heartbeat from core client for 30 sec - exiting 16:27:20 (2632): No heartbeat from core client for 30 sec - exiting 16:27:21 (2632): No heartbeat from core client for 30 sec - exiting 16:27:23 (2632): No heartbeat from core client for 30 sec - exiting 16:27:24 (2632): No heartbeat from core client for 30 sec - exiting 16:27:25 (2632): No heartbeat from core client for 30 sec - exiting 16:27:26 (2632): No heartbeat from core client for 30 sec - exiting 16:27:27 (2632): No heartbeat from core client for 30 sec - exiting 16:27:28 (2632): No heartbeat from core client for 30 sec - exiting 16:27:29 (2632): No heartbeat from core client for 30 sec - exiting 16:27:30 (2632): No heartbeat from core client for 30 sec - exiting 16:27:31 (2632): No heartbeat from core client for 30 sec - exiting 16:27:32 (2632): No heartbeat from core client for 30 sec - exiting 16:27:33 (2632): No heartbeat from core client for 30 sec - exiting 16:27:34 (2632): No heartbeat from core client for 30 sec - exiting 16:27:35 (2632): No heartbeat from core client for 30 sec - exiting 16:27:36 (2632): No heartbeat from core client for 30 sec - exiting 16:27:37 (2632): No heartbeat from core client for 30 sec - exiting 16:27:38 (2632): No heartbeat from core client for 30 sec - exiting 16:27:39 (2632): No heartbeat from core client for 30 sec - exiting 16:27:40 (2632): No heartbeat from core client for 30 sec - exiting 16:27:42 (2632): No heartbeat from core client for 30 sec - exiting 16:27:43 (2632): No heartbeat from core client for 30 sec - exiting 16:27:44 (2632): No heartbeat from core client for 30 sec - exiting 16:27:45 (2632): No heartbeat from core client for 30 sec - exiting 16:27:46 (2632): No heartbeat from core client for 30 sec - exiting 16:27:47 (2632): No heartbeat from core client for 30 sec - exiting 16:27:48 (2632): No heartbeat from core client for 30 sec - exiting 16:27:49 (2632): No heartbeat from core client for 30 sec - exiting 16:27:50 (2632): No heartbeat from core client for 30 sec - exiting 16:27:51 (2632): No heartbeat from core client for 30 sec - exiting 16:27:52 (2632): No heartbeat from core client for 30 sec - exiting 16:27:53 (2632): No heartbeat from core client for 30 sec - exiting 16:27:54 (2632): No heartbeat from core client for 30 sec - exiting 16:27:55 (2632): No heartbeat from core client for 30 sec - exiting 16:27:56 (2632): No heartbeat from core client for 30 sec - exiting 16:27:57 (2632): No heartbeat from core client for 30 sec - exiting 16:27:58 (2632): No heartbeat from core client for 30 sec - exiting 16:27:59 (2632): No heartbeat from core client for 30 sec - exiting 16:28:00 (2632): No heartbeat from core client for 30 sec - exiting 16:28:01 (2632): No heartbeat from core client for 30 sec - exiting 16:28:02 (2632): No heartbeat from core client for 30 sec - exiting 16:28:03 (2632): No heartbeat from core client for 30 sec - exiting 16:28:04 (2632): No heartbeat from core client for 30 sec - exiting 16:28:05 (2632): No heartbeat from core client for 30 sec - exiting 16:28:06 (2632): No heartbeat from core client for 30 sec - exiting 16:28:07 (2632): No heartbeat from core client for 30 sec - exiting 16:28:08 (2632): No heartbeat from core client for 30 sec - exiting 16:28:09 (2632): No heartbeat from core client for 30 sec - exiting 16:28:11 (2632): No heartbeat from core client for 30 sec - exiting 16:28:12 (2632): No heartbeat from core client for 30 sec - exiting 16:28:13 (2632): No heartbeat from core client for 30 sec - exiting 16:28:14 (2632): No heartbeat from core client for 30 sec - exiting 16:28:15 (2632): No heartbeat from core client for 30 sec - exiting 16:28:16 (2632): No heartbeat from core client for 30 sec - exiting 16:28:17 (2632): No heartbeat from core client for 30 sec - exiting 16:28:18 (2632): No heartbeat from core client for 30 sec - exiting 16:28:19 (2632): No heartbeat from core client for 30 sec - exiting 16:28:20 (2632): No heartbeat from core client for 30 sec - exiting 16:28:21 (2632): No heartbeat from core client for 30 sec - exiting 16:28:22 (2632): No heartbeat from core client for 30 sec - exiting 16:28:23 (2632): No heartbeat from core client for 30 sec - exiting 16:28:24 (2632): No heartbeat from core client for 30 sec - exiting 16:28:25 (2632): No heartbeat from core client for 30 sec - exiting 16:28:26 (2632): No heartbeat from core client for 30 sec - exiting 16:28:27 (2632): No heartbeat from core client for 30 sec - exiting 16:28:28 (2632): No heartbeat from core client for 30 sec - exiting 16:28:29 (2632): No heartbeat from core client for 30 sec - exiting 16:28:30 (2632): No heartbeat from core client for 30 sec - exiting 16:28:31 (2632): No heartbeat from core client for 30 sec - exiting 16:28:32 (2632): No heartbeat from core client for 30 sec - exiting 16:28:33 (2632): No heartbeat from core client for 30 sec - exiting 16:28:34 (2632): No heartbeat from core client for 30 sec - exiting 16:28:35 (2632): No heartbeat from core client for 30 sec - exiting 16:28:36 (2632): No heartbeat from core client for 30 sec - exiting 16:28:37 (2632): No heartbeat from core client for 30 sec - exiting 16:28:38 (2632): No heartbeat from core client for 30 sec - exiting 16:28:39 (2632): No heartbeat from core client for 30 sec - exiting 16:28:41 (2632): No heartbeat from core client for 30 sec - exiting 16:28:42 (2632): No heartbeat from core client for 30 sec - exiting 16:28:43 (2632): No heartbeat from core client for 30 sec - exiting 16:28:44 (2632): No heartbeat from core client for 30 sec - exiting 16:28:45 (2632): No heartbeat from core client for 30 sec - exiting 16:28:46 (2632): No heartbeat from core client for 30 sec - exiting 16:28:47 (2632): No heartbeat from core client for 30 sec - exiting 16:28:48 (2632): No heartbeat from core client for 30 sec - exiting 16:28:49 (2632): No heartbeat from core client for 30 sec - exiting 16:28:50 (2632): No heartbeat from core client for 30 sec - exiting 16:28:51 (2632): No heartbeat from core client for 30 sec - exiting 16:28:52 (2632): No heartbeat from core client for 30 sec - exiting 16:28:53 (2632): No heartbeat from core client for 30 sec - exiting 16:28:54 (2632): No heartbeat from core client for 30 sec - exiting 16:28:55 (2632): No heartbeat from core client for 30 sec - exiting 16:28:56 (2632): No heartbeat from core client for 30 sec - exiting 16:28:57 (2632): No heartbeat from core client for 30 sec - exiting 16:28:58 (2632): No heartbeat from core client for 30 sec - exiting 16:28:59 (2632): No heartbeat from core client for 30 sec - exiting 16:29:00 (2632): No heartbeat from core client for 30 sec - exiting 16:29:01 (2632): No heartbeat from core client for 30 sec - exiting 16:29:02 (2632): No heartbeat from core client for 30 sec - exiting 16:29:03 (2632): No heartbeat from core client for 30 sec - exiting 16:29:04 (2632): No heartbeat from core client for 30 sec - exiting 16:29:05 (2632): No heartbeat from core client for 30 sec - exiting 16:29:06 (2632): No heartbeat from core client for 30 sec - exiting 16:29:07 (2632): No heartbeat from core client for 30 sec - exiting 16:29:08 (2632): No heartbeat from core client for 30 sec - exiting 16:29:09 (2632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:15 (5700): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5548, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 16:47:11 (6080): No heartbeat from core client for 30 sec - exiting 16:47:12 (6080): No heartbeat from core client for 30 sec - exiting 16:47:13 (6080): No heartbeat from core client for 30 sec - exiting 16:47:14 (6080): No heartbeat from core client for 30 sec - exiting 16:47:15 (6080): No heartbeat from core client for 30 sec - exiting 16:47:16 (6080): No heartbeat from core client for 30 sec - exiting 16:47:17 (6080): No heartbeat from core client for 30 sec - exiting 16:47:18 (6080): No heartbeat from core client for 30 sec - exiting 16:47:19 (6080): No heartbeat from core client for 30 sec - exiting 16:47:20 (6080): No heartbeat from core client for 30 sec - exiting 16:47:21 (6080): No heartbeat from core client for 30 sec - exiting 16:47:22 (6080): No heartbeat from core client for 30 sec - exiting 16:47:23 (6080): No heartbeat from core client for 30 sec - exiting 16:47:24 (6080): No heartbeat from core client for 30 sec - exiting 16:47:25 (6080): No heartbeat from core client for 30 sec - exiting 16:47:26 (6080): No heartbeat from core client for 30 sec - exiting 16:47:27 (6080): No heartbeat from core client for 30 sec - exiting 16:47:28 (6080): No heartbeat from core client for 30 sec - exiting 16:47:29 (6080): No heartbeat from core client for 30 sec - exiting 16:47:30 (6080): No heartbeat from core client for 30 sec - exiting 16:47:31 (6080): No heartbeat from core client for 30 sec - exiting 16:47:32 (6080): No heartbeat from core client for 30 sec - exiting 16:47:33 (6080): No heartbeat from core client for 30 sec - exiting 16:47:34 (6080): No heartbeat from core client for 30 sec - exiting 16:47:35 (6080): No heartbeat from core client for 30 sec - exiting 16:47:36 (6080): No heartbeat from core client for 30 sec - exiting 16:47:37 (6080): No heartbeat from core client for 30 sec - exiting 16:47:38 (6080): No heartbeat from core client for 30 sec - exiting 16:47:39 (6080): No heartbeat from core client for 30 sec - exiting 16:47:40 (6080): No heartbeat from core client for 30 sec - exiting 16:47:41 (6080): No heartbeat from core client for 30 sec - exiting 16:47:42 (6080): No heartbeat from core client for 30 sec - exiting 16:47:43 (6080): No heartbeat from core client for 30 sec - exiting 16:47:44 (6080): No heartbeat from core client for 30 sec - exiting 16:47:45 (6080): No heartbeat from core client for 30 sec - exiting 16:47:46 (6080): No heartbeat from core client for 30 sec - exiting 16:47:47 (6080): No heartbeat from core client for 30 sec - exiting 16:47:48 (6080): No heartbeat from core client for 30 sec - exiting 16:47:49 (6080): No heartbeat from core client for 30 sec - exiting 16:47:50 (6080): No heartbeat from core client for 30 sec - exiting 16:47:51 (6080): No heartbeat from core client for 30 sec - exiting 16:47:52 (6080): No heartbeat from core client for 30 sec - exiting 16:47:53 (6080): No heartbeat from core client for 30 sec - exiting 16:47:54 (6080): No heartbeat from core client for 30 sec - exiting 16:47:55 (6080): No heartbeat from core client for 30 sec - exiting 16:47:56 (6080): No heartbeat from core client for 30 sec - exiting 16:47:57 (6080): No heartbeat from core client for 30 sec - exiting 16:47:58 (6080): No heartbeat from core client for 30 sec - exiting 16:47:59 (6080): No heartbeat from core client for 30 sec - exiting 16:48:01 (6080): No heartbeat from core client for 30 sec - exiting 16:48:02 (6080): No heartbeat from core client for 30 sec - exiting 16:48:03 (6080): No heartbeat from core client for 30 sec - exiting 16:48:04 (6080): No heartbeat from core client for 30 sec - exiting 16:48:05 (6080): No heartbeat from core client for 30 sec - exiting 16:48:06 (6080): No heartbeat from core client for 30 sec - exiting 16:48:07 (6080): No heartbeat from core client for 30 sec - exiting 16:48:08 (6080): No heartbeat from core client for 30 sec - exiting 16:48:09 (6080): No heartbeat from core client for 30 sec - exiting 16:48:10 (6080): No heartbeat from core client for 30 sec - exiting 16:48:11 (6080): No heartbeat from core client for 30 sec - exiting 16:48:12 (6080): No heartbeat from core client for 30 sec - exiting 16:48:13 (6080): No heartbeat from core client for 30 sec - exiting 16:48:14 (6080): No heartbeat from core client for 30 sec - exiting 16:48:15 (6080): No heartbeat from core client for 30 sec - exiting 16:48:16 (6080): No heartbeat from core client for 30 sec - exiting 16:48:17 (6080): No heartbeat from core client for 30 sec - exiting 16:48:18 (6080): No heartbeat from core client for 30 sec - exiting 16:48:19 (6080): No heartbeat from core client for 30 sec - exiting 16:48:20 (6080): No heartbeat from core client for 30 sec - exiting 16:48:21 (6080): No heartbeat from core client for 30 sec - exiting 16:48:22 (6080): No heartbeat from core client for 30 sec - exiting 16:48:23 (6080): No heartbeat from core client for 30 sec - exiting 16:48:24 (6080): No heartbeat from core client for 30 sec - exiting 16:48:25 (6080): No heartbeat from core client for 30 sec - exiting 16:48:26 (6080): No heartbeat from core client for 30 sec - exiting 16:48:27 (6080): No heartbeat from core client for 30 sec - exiting 16:48:28 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4340, selfPID=4340, iMonCtr=2 08:23:34 (3276): No heartbeat from core client for 30 sec - exiting 08:23:35 (3276): No heartbeat from core client for 30 sec - exiting 08:23:36 (3276): No heartbeat from core client for 30 sec - exiting 08:23:37 (3276): No heartbeat from core client for 30 sec - exiting 08:23:38 (3276): No heartbeat from core client for 30 sec - exiting 08:23:39 (3276): No heartbeat from core client for 30 sec - exiting 08:23:40 (3276): No heartbeat from core client for 30 sec - exiting 08:23:41 (3276): No heartbeat from core client for 30 sec - exiting 08:23:42 (3276): No heartbeat from core client for 30 sec - exiting 08:23:43 (3276): No heartbeat from core client for 30 sec - exiting 08:23:44 (3276): No heartbeat from core client for 30 sec - exiting 08:23:45 (3276): No heartbeat from core client for 30 sec - exiting 08:23:46 (3276): No heartbeat from core client for 30 sec - exiting 08:23:47 (3276): No heartbeat from core client for 30 sec - exiting 08:23:48 (3276): No heartbeat from core client for 30 sec - exiting 08:23:49 (3276): No heartbeat from core client for 30 sec - exiting 08:23:50 (3276): No heartbeat from core client for 30 sec - exiting 08:23:52 (3276): No heartbeat from core client for 30 sec - exiting 08:23:53 (3276): No heartbeat from core client for 30 sec - exiting 08:23:54 (3276): No heartbeat from core client for 30 sec - exiting 08:23:55 (3276): No heartbeat from core client for 30 sec - exiting 08:23:56 (3276): No heartbeat from core client for 30 sec - exiting 08:23:57 (3276): No heartbeat from core client for 30 sec - exiting 08:23:58 (3276): No heartbeat from core client for 30 sec - exiting 08:23:59 (3276): No heartbeat from core client for 30 sec - exiting 08:24:00 (3276): No heartbeat from core client for 30 sec - exiting 08:24:01 (3276): No heartbeat from core client for 30 sec - exiting 08:24:02 (3276): No heartbeat from core client for 30 sec - exiting 08:24:03 (3276): No heartbeat from core client for 30 sec - exiting 08:24:04 (3276): No heartbeat from core client for 30 sec - exiting 08:24:05 (3276): No heartbeat from core client for 30 sec - exiting 08:24:06 (3276): No heartbeat from core client for 30 sec - exiting 08:24:07 (3276): No heartbeat from core client for 30 sec - exiting 08:24:08 (3276): No heartbeat from core client for 30 sec - exiting 08:24:09 (3276): No heartbeat from core client for 30 sec - exiting 08:24:10 (3276): No heartbeat from core client for 30 sec - exiting 08:24:11 (3276): No heartbeat from core client for 30 sec - exiting 08:24:12 (3276): No heartbeat from core client for 30 sec - exiting 08:24:13 (3276): No heartbeat from core client for 30 sec - exiting 08:24:14 (3276): No heartbeat from core client for 30 sec - exiting 08:24:15 (3276): No heartbeat from core client for 30 sec - exiting 08:24:16 (3276): No heartbeat from core client for 30 sec - exiting 08:24:17 (3276): No heartbeat from core client for 30 sec - exiting 08:24:18 (3276): No heartbeat from core client for 30 sec - exiting 08:24:19 (3276): No heartbeat from core client for 30 sec - exiting 08:24:20 (3276): No heartbeat from core client for 30 sec - exiting 08:24:21 (3276): No heartbeat from core client for 30 sec - exiting 08:24:22 (3276): No heartbeat from core client for 30 sec - exiting 08:24:23 (3276): No heartbeat from core client for 30 sec - exiting 08:24:24 (3276): No heartbeat from core client for 30 sec - exiting 08:24:25 (3276): No heartbeat from core client for 30 sec - exiting 08:24:26 (3276): No heartbeat from core client for 30 sec - exiting 08:24:27 (3276): No heartbeat from core client for 30 sec - exiting 08:24:28 (3276): No heartbeat from core client for 30 sec - exiting 08:24:29 (3276): No heartbeat from core client for 30 sec - exiting 08:24:30 (3276): No heartbeat from core client for 30 sec - exiting 08:24:31 (3276): No heartbeat from core client for 30 sec - exiting 08:24:32 (3276): No heartbeat from core client for 30 sec - exiting 08:24:33 (3276): No heartbeat from core client for 30 sec - exiting 08:24:34 (3276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting,CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:35:25 (4476): start_timer_thread(): CreateThread() failed, errno 0 17:35:25 (5932): start_timer_thread(): CreateThread() failed, errno 0 20:12:34 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:23:23 (3100): No heartbeat from core client for 30 sec - exiting 20:23:25 (3100): No heartbeat from core client for 30 sec - exiting 20:23:26 (3100): No heartbeat from core client for 30 sec - exiting 20:23:27 (3100): No heartbeat from core client for 30 sec - exiting 20:23:28 (3100): No heartbeat from core client for 30 sec - exiting 20:23:29 (3100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:55:51 (3184): start_timer_thread(): CreateThread() failed, errno 0 19:55:51 (4228): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5520, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3888, selfPID=3888, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:15:35 (2484): start_timer_thread(): CreateThread() failed, errno 0 17:15:35 (2672): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2672, selfPID=2672, iMonCtr=2 18:44:03 (2308): start_timer_thread(): CreateThread() failed, errno 0 18:44:03 (964): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:36:21 (5292): start_timer_thread(): CreateThread() failed, errno 0 21:36:21 (2268): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 09:49:39 (4092): No heartbeat from core client for 30 sec - exiting 09:49:40 (4092): No heartbeat from core client for 30 sec - exiting 09:49:41 (4092): No heartbeat from core client for 30 sec - exiting 09:49:42 (4092): No heartbeat from core client for 30 sec - exiting 09:49:43 (4092): No heartbeat from core client for 30 sec - exiting 09:49:44 (4092): No heartbeat from core client for 30 sec - exiting 09:49:45 (4092): No heartbeat from core client for 30 sec - exiting 09:49:46 (4092): No heartbeat from core client for 30 sec - exiting 09:49:47 (4092): No heartbeat from core client for 30 sec - exiting 09:49:48 (4092): No heartbeat from core client for 30 sec - exiting 09:49:49 (4092): No heartbeat from core client for 30 sec - exiting 09:49:50 (4092): No heartbeat from core client for 30 sec - exiting 09:49:51 (4092): No heartbeat from core client for 30 sec - exiting 09:49:52 (4092): No heartbeat from core client for 30 sec - exiting 09:49:53 (4092): No heartbeat from core client for 30 sec - exiting 09:49:55 (4092): No heartbeat from core client for 30 sec - exiting 09:49:56 (4092): No heartbeat from core client for 30 sec - exiting 09:49:57 (4092): No heartbeat from core client for 30 sec - exiting 09:49:58 (4092): No heartbeat from core client for 30 sec - exiting 09:49:59 (4092): No heartbeat from core client for 30 sec - exiting 09:50:00 (4092): No heartbeat from core client for 30 sec - exiting 09:50:01 (4092): No heartbeat from core client for 30 sec - exiting 09:50:02 (4092): No heartbeat from core client for 30 sec - exiting 09:50:03 (4092): No heartbeat from core client for 30 sec - exiting 09:50:04 (4092): No heartbeat from core client for 30 sec - exiting 09:50:05 (4092): No heartbeat from core client for 30 sec - exiting 09:50:06 (4092): No heartbeat from core client for 30 sec - exiting 09:50:07 (4092): No heartbeat from core client for 30 sec - exiting 09:50:08 (4092): No heartbeat from core client for 30 sec - exiting 09:50:09 (4092): No heartbeat from core client for 30 sec - exiting 09:50:10 (4092): No heartbeat from core client for 30 sec - exiting 09:50:11 (4092): No heartbeat from core client for 30 sec - exiting 09:50:12 (4092): No heartbeat from core client for 30 sec - exiting 09:50:13 (4092): No heartbeat from core client for 30 sec - exiting 09:50:14 (4092): No heartbeat from core client for 30 sec - exiting 09:50:15 (4092): No heartbeat from core client for 30 sec - exiting 09:50:16 (4092): No heartbeat from core client for 30 sec - exiting 09:50:17 (4092): No heartbeat from core client for 30 sec - exiting 09:50:18 (4092): No heartbeat from core client for 30 sec - exiting 09:50:19 (4092): No heartbeat from core client for 30 sec - exiting 09:50:20 (4092): No heartbeat from core client for 30 sec - exiting 09:50:21 (4092): No heartbeat from core client for 30 sec - exiting 09:50:22 (4092): No heartbeat from core client for 30 sec - exiting 09:50:23 (4092): No heartbeat from core client for 30 sec - exiting 09:50:24 (4092): No heartbeat from core client for 30 sec - exiting 09:50:25 (4092): No heartbeat from core client for 30 sec - exiting 09:50:26 (4092): No heartbeat from core client for 30 sec - exiting 09:50:27 (4092): No heartbeat from core client for 30 sec - exiting 09:50:28 (4092): No heartbeat from core client for 30 sec - exiting 09:50:29 (4092): No heartbeat from core client for 30 sec - exiting 09:50:30 (4092): No heartbeat from core client for 30 sec - exiting 09:50:31 (4092): No heartbeat from core client for 30 sec - exiting 09:50:32 (4092): No heartbeat from core client for 30 sec - exiting 09:50:33 (4092): No heartbeat from core client for 30 sec - exiting 09:50:34 (4092): No heartbeat from core client for 30 sec - exiting 09:50:35 (4092): No heartbeat from core client for 30 sec - exiting 09:50:36 (4092): No heartbeat from core client for 30 sec - exiting 09:50:37 (4092): No heartbeat from core client for 30 sec - exiting 09:50:38 (4092): No heartbeat from core client for 30 sec - exiting 09:50:39 (4092): No heartbeat from core client for 30 sec - exiting 09:50:40 (4092): No heartbeat from core client for 30 sec - exiting 09:50:41 (4092): No heartbeat from core client for 30 sec - exiting 09:50:42 (4092): No heartbeat from core client for 30 sec - exiting 09:50:43 (4092): No heartbeat from core client for 30 sec - exiting 09:50:44 (4092): No heartbeat from core client for 30 sec - exiting 09:50:45 (4092): No heartbeat from core client for 30 sec - exiting 09:50:46 (4092): No heartbeat from core client for 30 sec - exiting 09:50:47 (4092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5772, selfPID=5772, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 16:48:09 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:41 (2588): No heartbeat from core client for 30 sec - exiting 18:36:42 (2588): No heartbeat from core client for 30 sec - exiting 18:36:43 (2588): No heartbeat from core client for 30 sec - exiting 18:36:44 (2588): No heartbeat from core client for 30 sec - exiting 18:36:45 (2588): No heartbeat from core client for 30 sec - exiting 18:36:46 (2588): No heartbeat from core client for 30 sec - exiting 18:36:47 (2588): No heartbeat from core client for 30 sec - exiting 18:36:48 (2588): No heartbeat from core client for 30 sec - exiting 18:36:49 (2588): No heartbeat from core client for 30 sec - exiting 18:36:50 (2588): No heartbeat from core client for 30 sec - exiting 18:36:51 (2588): No heartbeat from core client for 30 sec - exiting 18:36:52 (2588): No heartbeat from core client for 30 sec - exiting 18:36:53 (2588): No heartbeat from core client for 30 sec - exiting 18:36:54 (2588): No heartbeat from core client for 30 sec - exiting 18:36:55 (2588): No heartbeat from core client for 30 sec - exiting 18:36:56 (2588): No heartbeat from core client for 30 sec - exiting 18:36:57 (2588): No heartbeat from core client for 30 sec - exiting 18:36:58 (2588): No heartbeat from core client for 30 sec - exiting 18:36:59 (2588): No heartbeat from core client for 30 sec - exiting 18:37:00 (2588): No heartbeat from core client for 30 sec - exiting 18:37:01 (2588): No heartbeat from core client for 30 sec - exiting 18:37:02 (2588): No heartbeat from core client for 30 sec - exiting 18:37:03 (2588): No heartbeat from core client for 30 sec - exiting 18:37:04 (2588): No heartbeat from core client for 30 sec - exiting 18:37:05 (2588): No heartbeat from core client for 30 sec - exiting 18:37:06 (2588): No heartbeat from core client for 30 sec - exiting 18:37:07 (2588): No heartbeat from core client for 30 sec - exiting 18:37:08 (2588): No heartbeat from core client for 30 sec - exiting 18:37:09 (2588): No heartbeat from core client for 30 sec - exiting 18:37:10 (2588): No heartbeat from core client for 30 sec - exiting 18:37:11 (2588): No heartbeat from core client for 30 sec - exiting 18:37:12 (2588): No heartbeat from core client for 30 sec - exiting 18:37:14 (2588): No heartbeat from core client for 30 sec - exiting 18:37:15 (2588): No heartbeat from core client for 30 sec - exiting 18:37:16 (2588): No heartbeat from core client for 30 sec - exiting 18:37:17 (2588): No heartbeat from core client for 30 sec - exiting 18:37:18 (2588): No heartbeat from core client for 30 sec - exiting 18:37:19 (2588): No heartbeat from core client for 30 sec - exiting 18:37:20 (2588): No heartbeat from core client for 30 sec - exiting 18:37:21 (2588): No heartbeat from core client for 30 sec - exiting 18:37:22 (2588): No heartbeat from core client for 30 sec - exiting 18:37:23 (2588): No heartbeat from core client for 30 sec - exiting 18:37:24 (2588): No heartbeat from core client for 30 sec - exiting 18:37:25 (2588): No heartbeat from core client for 30 sec - exiting 18:37:26 (2588): No heartbeat from core client for 30 sec - exiting 18:37:27 (2588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:19:12 (4472): No heartbeat from core client for 30 sec - exiting 06:19:13 (4472): No heartbeat from core client for 30 sec - exiting 06:19:14 (4472): No heartbeat from core client for 30 sec - exiting 06:19:15 (4472): No heartbeat from core client for 30 sec - exiting 06:19:16 (4472): No heartbeat from core client for 30 sec - exiting 06:19:17 (4472): No heartbeat from core client for 30 sec - exiting 06:19:18 (4472): No heartbeat from core client for 30 sec - exiting 06:19:19 (4472): No heartbeat from core client for 30 sec - exiting 06:19:20 (4472): No heartbeat from core client for 30 sec - exiting 06:19:21 (4472): No heartbeat from core client for 30 sec - exiting 06:19:22 (4472): No heartbeat from core client for 30 sec - exiting 06:19:23 (4472): No heartbeat from core client for 30 sec - exiting 06:19:24 (4472): No heartbeat from core client for 30 sec - exiting 06:19:25 (4472): No heartbeat from core client for 30 sec - exiting 06:19:26 (4472): No heartbeat from core client for 30 sec - exiting 06:19:27 (4472): No heartbeat from core client for 30 sec - exiting 06:19:28 (4472): No heartbeat from core client for 30 sec - exiting 06:19:29 (4472): No heartbeat from core client for 30 sec - exiting 06:19:30 (4472): No heartbeat from core client for 30 sec - exiting 06:19:31 (4472): No heartbeat from core client for 30 sec - exiting 06:19:32 (4472): No heartbeat from core client for 30 sec - exiting 06:19:33 (4472): No heartbeat from core client for 30 sec - exiting 06:19:34 (4472): No heartbeat from core client for 30 sec - exiting 06:19:35 (4472): No heartbeat from core client for 30 sec - exiting 06:19:36 (4472): No heartbeat from core client for 30 sec - exiting 06:19:37 (4472): No heartbeat from core client for 30 sec - exiting 06:19:38 (4472): No heartbeat from core client for 30 sec - exiting 06:19:39 (4472): No heartbeat from core client for 30 sec - exiting 06:19:40 (4472): No heartbeat from core client for 30 sec - exiting 06:19:41 (4472): No heartbeat from core client for 30 sec - exiting 06:19:42 (4472): No heartbeat from core client for 30 sec - exiting 06:19:43 (4472): No heartbeat from core client for 30 sec - exiting 06:19:44 (4472): No heartbeat from core client for 30 sec - exiting 06:19:45 (4472): No heartbeat from core client for 30 sec - exiting 06:19:46 (4472): No heartbeat from core client for 30 sec - exiting 06:19:47 (4472): No heartbeat from core client for 30 sec - exiting 06:19:48 (4472): No heartbeat from core client for 30 sec - exiting 06:19:49 (4472): No heartbeat from core client for 30 sec - exiting 06:19:51 (4472): No heartbeat from core client for 30 sec - exiting 06:19:52 (4472): No heartbeat from core client for 30 sec - exiting 06:19:53 (4472): No heartbeat from core client for 30 sec - exiting 06:19:54 (4472): No heartbeat from core client for 30 sec - exiting 06:19:55 (4472): No heartbeat from core client for 30 sec - exiting 06:19:56 (4472): No heartbeat from core client for 30 sec - exiting 06:19:57 (4472): No heartbeat from core client for 30 sec - exiting 06:19:58 (4472): No heartbeat from core client for 30 sec - exiting 06:19:59 (4472): No heartbeat from core client for 30 sec - exiting 06:20:00 (4472): No heartbeat from core client for 30 sec - exiting 06:20:01 (4472): No heartbeat from core client for 30 sec - exiting 06:20:02 (4472): No heartbeat from core client for 30 sec - exiting 06:20:03 (4472): No heartbeat from core client for 30 sec - exiting 06:20:04 (4472): No heartbeat from core client for 30 sec - exiting 06:20:05 (4472): No heartbeat from core client for 30 sec - exiting 06:20:06 (4472): No heartbeat from core client for 30 sec - exiting 06:20:07 (4472): No heartbeat from core client for 30 sec - exiting 06:20:08 (4472): No heartbeat from core client for 30 sec - exiting 06:20:09 (4472): No heartbeat from core client for 30 sec - exiting 06:20:10 (4472): No heartbeat from core client for 30 sec - exiting 06:20:11 (4472): No heartbeat from core client for 30 sec - exiting 06:20:12 (4472): No heartbeat from core client for 30 sec - exiting 06:20:13 (4472): No heartbeat from core client for 30 sec - exiting 06:20:14 (4472): No heartbeat from core client for 30 sec - exiting 06:20:15 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4272, selfPID=4272, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3060, selfPID=3060, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4700, selfPID=4700, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4372, selfPID=4372, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 06:27:16 (3736): No heartbeat from core client for 30 sec - exiting 06:27:17 (3736): No heartbeat from core client for 30 sec - exiting 06:27:18 (3736): No heartbeat from core client for 30 sec - exiting 06:27:19 (3736): No heartbeat from core client for 30 sec - exiting 06:27:20 (3736): No heartbeat from core client for 30 sec - exiting 06:27:21 (3736): No heartbeat from core client for 30 sec - exiting 06:27:22 (3736): No heartbeat from core client for 30 sec - exiting 06:27:23 (3736): No heartbeat from core client for 30 sec - exiting 06:27:24 (3736): No heartbeat from core client for 30 sec - exiting 06:27:25 (3736): No heartbeat from core client for 30 sec - exiting 06:27:26 (3736): No heartbeat from core client for 30 sec - exiting 06:27:27 (3736): No heartbeat from core client for 30 sec - exiting 06:27:28 (3736): No heartbeat from core client for 30 sec - exiting 06:27:29 (3736): No heartbeat from core client for 30 sec - exiting 06:27:30 (3736): No heartbeat from core client for 30 sec - exiting 06:27:31 (3736): No heartbeat from core client for 30 sec - exiting 06:27:32 (3736): No heartbeat from core client for 30 sec - exiting 06:27:33 (3736): No heartbeat from core client for 30 sec - exiting 06:27:34 (3736): No heartbeat from core client for 30 sec - exiting 06:27:35 (3736): No heartbeat from core client for 30 sec - exiting 06:27:36 (3736): No heartbeat from core client for 30 sec - exiting 06:27:37 (3736): No heartbeat from core client for 30 sec - exiting 06:27:38 (3736): No heartbeat from core client for 30 sec - exiting 06:27:39 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:32:03 (3100): start_timer_thread(): CreateThread() failed, errno 0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1140, selfPID=2056, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 May 2011 17:37:41	939489	12791047	hadam3p_pnw_z7z1_1972_1_006940853_2	46,176	242,533	5.2524
14 May 2011 12:27:57	939489	12791047	hadam3p_pnw_z7z1_1972_1_006940853_2	34,656	182,815	5.2751
08 May 2011 17:42:07	939489	12791047	hadam3p_pnw_z7z1_1972_1_006940853_2	23,136	123,104	5.3209
24 Apr 2011 10:14:20	939489	12791047	hadam3p_pnw_z7z1_1972_1_006940853_2	11,616	62,652	5.3936