Task 14023441

Name	hadcm3n_y8vr_1940_40_007742229_4
Workunit	7897337
Created	28 Jan 2012, 10:51:38 UTC
Sent	28 Jan 2012, 10:51:43 UTC
Report deadline	28 Apr 2012, 18:18:54 UTC
Received	12 Feb 2012, 5:12:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1020546
Run time	9 days 10 hours 44 min 3 sec
CPU time	9 days 0 hours 48 min 36 sec
Validate state	Invalid
Credit	4,665.60
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1284, iMonCtr=1 Model crash detected, will try to restart... 03:15:01 (7972): No heartbeat from core client for 30 sec - exiting 03:15:02 (7972): No heartbeat from core client for 30 sec - exiting 03:15:03 (7972): No heartbeat from core client for 30 sec - exiting 03:15:04 (7972): No heartbeat from core client for 30 sec - exiting 03:15:05 (7972): No heartbeat from core client for 30 sec - exiting 03:15:06 (7972): No heartbeat from core client for 30 sec - exiting 03:15:07 (7972): No heartbeat from core client for 30 sec - exiting 03:15:08 (7972): No heartbeat from core client for 30 sec - exiting 03:15:09 (7972): No heartbeat from core client for 30 sec - exiting 03:15:10 (7972): No heartbeat from core client for 30 sec - exiting 03:15:11 (7972): No heartbeat from core client for 30 sec - exiting 03:15:12 (7972): No heartbeat from core client for 30 sec - exiting 03:15:13 (7972): No heartbeat from core client for 30 sec - exiting 03:15:14 (7972): No heartbeat from core client for 30 sec - exiting 03:15:15 (7972): No heartbeat from core client for 30 sec - exiting 03:15:16 (7972): No heartbeat from core client for 30 sec - exiting 03:15:17 (7972): No heartbeat from core client for 30 sec - exiting 03:15:18 (7972): No heartbeat from core client for 30 sec - exiting 03:15:19 (7972): No heartbeat from core client for 30 sec - exiting 03:15:20 (7972): No heartbeat from core client for 30 sec - exiting 03:15:21 (7972): No heartbeat from core client for 30 sec - exiting 03:15:22 (7972): No heartbeat from core client for 30 sec - exiting 03:15:23 (7972): No heartbeat from core client for 30 sec - exiting 03:15:24 (7972): No heartbeat from core client for 30 sec - exiting 03:15:25 (7972): No heartbeat from core client for 30 sec - exiting 03:15:26 (7972): No heartbeat from core client for 30 sec - exiting 03:15:27 (7972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:03:56 (8996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:04:08 (8996): No heartbeat from core client for 30 sec - exiting 18:04:09 (8996): No heartbeat from core client for 30 sec - exiting 18:04:11 (8996): No heartbeat from core client for 30 sec - exiting 18:04:13 (8996): No heartbeat from core client for 30 sec - exiting 18:13:50 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:14:30 (9452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:03:00 (1544): No heartbeat from core client for 30 sec - exiting 05:03:01 (1544): No heartbeat from core client for 30 sec - exiting 05:03:02 (1544): No heartbeat from core client for 30 sec - exiting 05:03:03 (1544): No heartbeat from core client for 30 sec - exiting 05:03:04 (1544): No heartbeat from core client for 30 sec - exiting 05:03:05 (1544): No heartbeat from core client for 30 sec - exiting 05:03:06 (1544): No heartbeat from core client for 30 sec - exiting 05:03:07 (1544): No heartbeat from core client for 30 sec - exiting 05:03:08 (1544): No heartbeat from core client for 30 sec - exiting 05:03:09 (1544): No heartbeat from core client for 30 sec - exiting 05:03:10 (1544): No heartbeat from core client for 30 sec - exiting 05:03:11 (1544): No heartbeat from core client for 30 sec - exiting 05:03:12 (1544): No heartbeat from core client for 30 sec - exiting 05:03:13 (1544): No heartbeat from core client for 30 sec - exiting 05:03:14 (1544): No heartbeat from core client for 30 sec - exiting 05:03:15 (1544): No heartbeat from core client for 30 sec - exiting 05:03:16 (1544): No heartbeat from core client for 30 sec - exiting 05:03:17 (1544): No heartbeat from core client for 30 sec - exiting 05:03:18 (1544): No heartbeat from core client for 30 sec - exiting 05:03:19 (1544): No heartbeat from core client for 30 sec - exiting 05:03:20 (1544): No heartbeat from core client for 30 sec - exiting 05:03:21 (1544): No heartbeat from core client for 30 sec - exiting 05:03:22 (1544): No heartbeat from core client for 30 sec - exiting 05:03:23 (1544): No heartbeat from core client for 30 sec - exiting 05:03:24 (1544): No heartbeat from core client for 30 sec - exiting 05:03:25 (1544): No heartbeat from core client for 30 sec - exiting 05:03:26 (1544): No heartbeat from core client for 30 sec - exiting 05:03:27 (1544): No heartbeat from core client for 30 sec - exiting 05:03:28 (1544): No heartbeat from core client for 30 sec - exiting 05:03:29 (1544): No heartbeat from core client for 30 sec - exiting 05:03:30 (1544): No heartbeat from core client for 30 sec - exiting 05:03:31 (1544): No heartbeat from core client for 30 sec - exiting 05:03:32 (1544): No heartbeat from core client for 30 sec - exiting 05:03:33 (1544): No heartbeat from core client for 30 sec - exiting 05:03:34 (1544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:35 (1544): No heartbeat from core client for 30 sec - exiting 05:03:36 (1544): No heartbeat from core client for 30 sec - exiting 05:04:14 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:47:50 (4348): No heartbeat from core client for 30 sec - exiting 02:47:51 (4348): No heartbeat from core client for 30 sec - exiting 02:47:52 (4348): No heartbeat from core client for 30 sec - exiting 02:47:53 (4348): No heartbeat from core client for 30 sec - exiting 02:47:54 (4348): No heartbeat from core client for 30 sec - exiting 02:47:55 (4348): No heartbeat from core client for 30 sec - exiting 02:47:56 (4348): No heartbeat from core client for 30 sec - exiting 02:47:57 (4348): No heartbeat from core client for 30 sec - exiting 02:47:58 (4348): No heartbeat from core client for 30 sec - exiting 02:47:59 (4348): No heartbeat from core client for 30 sec - exiting 02:48:00 (4348): No heartbeat from core client for 30 sec - exiting 02:48:01 (4348): No heartbeat from core client for 30 sec - exiting 02:48:02 (4348): No heartbeat from core client for 30 sec - exiting 02:48:03 (4348): No heartbeat from core client for 30 sec - exiting 02:48:04 (4348): No heartbeat from core client for 30 sec - exiting 02:48:05 (4348): No heartbeat from core client for 30 sec - exiting 02:48:06 (4348): No heartbeat from core client for 30 sec - exiting 02:48:07 (4348): No heartbeat from core client for 30 sec - exiting 02:48:08 (4348): No heartbeat from core client for 30 sec - exiting 02:48:09 (4348): No heartbeat from core client for 30 sec - exiting 02:48:10 (4348): No heartbeat from core client for 30 sec - exiting 02:48:11 (4348): No heartbeat from core client for 30 sec - exiting 02:48:12 (4348): No heartbeat from core client for 30 sec - exiting 02:48:13 (4348): No heartbeat from core client for 30 sec - exiting 02:48:14 (4348): No heartbeat from core client for 30 sec - exiting 02:48:15 (4348): No heartbeat from core client for 30 sec - exiting 02:48:16 (4348): No heartbeat from core client for 30 sec - exiting 02:48:17 (4348): No heartbeat from core client for 30 sec - exiting 02:48:18 (4348): No heartbeat from core client for 30 sec - exiting 02:48:19 (4348): No heartbeat from core client for 30 sec - exiting 02:48:20 (4348): No heartbeat from core client for 30 sec - exiting 02:48:21 (4348): No heartbeat from core client for 30 sec - exiting 02:48:22 (4348): No heartbeat from core client for 30 sec - exiting 02:48:23 (4348): No heartbeat from core client for 30 sec - exiting 02:48:24 (4348): No heartbeat from core client for 30 sec - exiting 02:48:25 (4348): No heartbeat from core client for 30 sec - exiting 02:48:26 (4348): No heartbeat from core client for 30 sec - exiting 02:48:27 (4348): No heartbeat from core client for 30 sec - exiting 02:48:28 (4348): No heartbeat from core client for 30 sec - exiting 02:48:29 (4348): No heartbeat from core client for 30 sec - exiting 02:48:30 (4348): No heartbeat from core client for 30 sec - exiting 02:48:31 (4348): No heartbeat from core client for 30 sec - exiting 02:48:32 (4348): No heartbeat from core client for 30 sec - exiting 02:48:33 (4348): No heartbeat from core client for 30 sec - exiting 02:48:34 (4348): No heartbeat from core client for 30 sec - exiting 02:48:35 (4348): No heartbeat from core client for 30 sec - exiting 02:48:36 (4348): No heartbeat from core client for 30 sec - exiting 02:48:37 (4348): No heartbeat from core client for 30 sec - exiting 02:48:38 (4348): No heartbeat from core client for 30 sec - exiting 02:48:39 (4348): No heartbeat from core client for 30 sec - exiting 02:48:40 (4348): No heartbeat from core client for 30 sec - exiting 02:48:41 (4348): No heartbeat from core client for 30 sec - exiting 02:48:42 (4348): No heartbeat from core client for 30 sec - exiting 02:48:43 (4348): No heartbeat from core client for 30 sec - exiting 02:48:44 (4348): No heartbeat from core client for 30 sec - exiting 02:48:45 (4348): No heartbeat from core client for 30 sec - exiting 02:48:46 (4348): No heartbeat from core client for 30 sec - exiting 02:48:47 (4348): No heartbeat from core client for 30 sec - exiting 02:48:48 (4348): No heartbeat from core client for 30 sec - exiting 02:48:49 (4348): No heartbeat from core client for 30 sec - exiting 02:48:50 (4348): No heartbeat from core client for 30 sec - exiting 02:48:51 (4348): No heartbeat from core client for 30 sec - exiting 02:48:52 (4348): No heartbeat from core client for 30 sec - exiting 02:48:53 (4348): No heartbeat from core client for 30 sec - exiting 02:48:54 (4348): No heartbeat from core client for 30 sec - exiting 02:48:55 (4348): No heartbeat from core client for 30 sec - exiting 02:48:56 (4348): No heartbeat from core client for 30 sec - exiting 02:48:57 (4348): No heartbeat from core client for 30 sec - exiting 02:48:58 (4348): No heartbeat from core client for 30 sec - exiting 02:48:59 (4348): No heartbeat from core client for 30 sec - exiting 02:49:00 (4348): No heartbeat from core client for 30 sec - exiting 02:49:01 (4348): No heartbeat from core client for 30 sec - exiting 02:49:02 (4348): No heartbeat from core client for 30 sec - exiting 02:49:03 (4348): No heartbeat from core client for 30 sec - exiting 02:49:04 (4348): No heartbeat from core client for 30 sec - exiting 02:49:05 (4348): No heartbeat from core client for 30 sec - exiting 02:49:06 (4348): No heartbeat from core client for 30 sec - exiting 02:49:07 (4348): No heartbeat from core client for 30 sec - exiting 02:49:08 (4348): No heartbeat from core client for 30 sec - exiting 02:49:09 (4348): No heartbeat from core client for 30 sec - exiting 02:49:10 (4348): No heartbeat from core client for 30 sec - exiting 02:49:11 (4348): No heartbeat from core client for 30 sec - exiting 02:49:12 (4348): No heartbeat from core client for 30 sec - exiting 02:49:13 (4348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:53:18 (5728): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:21:36 (5976): No heartbeat from core client for 30 sec - exiting 08:21:37 (5976): No heartbeat from core client for 30 sec - exiting 08:21:38 (5976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:26:00 (7076): No heartbeat from core client for 30 sec - exiting 04:26:01 (7076): No heartbeat from core client for 30 sec - exiting 04:26:02 (7076): No heartbeat from core client for 30 sec - exiting 04:26:03 (7076): No heartbeat from core client for 30 sec - exiting 04:26:04 (7076): No heartbeat from core client for 30 sec - exiting 04:26:05 (7076): No heartbeat from core client for 30 sec - exiting 04:26:06 (7076): No heartbeat from core client for 30 sec - exiting 04:26:07 (7076): No heartbeat from core client for 30 sec - exiting 04:26:08 (7076): No heartbeat from core client for 30 sec - exiting 04:26:09 (7076): No heartbeat from core client for 30 sec - exiting 04:26:10 (7076): No heartbeat from core client for 30 sec - exiting 04:26:11 (7076): No heartbeat from core client for 30 sec - exiting 04:26:12 (7076): No heartbeat from core client for 30 sec - exiting 04:26:13 (7076): No heartbeat from core client for 30 sec - exiting 04:26:14 (7076): No heartbeat from core client for 30 sec - exiting 04:26:15 (7076): No heartbeat from core client for 30 sec - exiting 04:26:16 (7076): No heartbeat from core client for 30 sec - exiting 04:26:17 (7076): No heartbeat from core client for 30 sec - exiting 04:26:18 (7076): No heartbeat from core client for 30 sec - exiting 04:26:19 (7076): No heartbeat from core client for 30 sec - exiting 04:26:20 (7076): No heartbeat from core client for 30 sec - exiting 04:26:21 (7076): No heartbeat from core client for 30 sec - exiting 04:26:22 (7076): No heartbeat from core client for 30 sec - exiting 04:26:23 (7076): No heartbeat from core client for 30 sec - exiting 04:26:24 (7076): No heartbeat from core client for 30 sec - exiting 04:26:25 (7076): No heartbeat from core client for 30 sec - exiting 04:26:26 (7076): No heartbeat from core client for 30 sec - exiting 04:26:27 (7076): No heartbeat from core client for 30 sec - exiting 04:26:28 (7076): No heartbeat from core client for 30 sec - exiting 04:26:29 (7076): No heartbeat from core client for 30 sec - exiting 04:26:30 (7076): No heartbeat from core client for 30 sec - exiting 04:26:31 (7076): No heartbeat from core client for 30 sec - exiting 04:26:32 (7076): No heartbeat from core client for 30 sec - exiting 04:26:33 (7076): No heartbeat from core client for 30 sec - exiting 04:26:34 (7076): No heartbeat from core client for 30 sec - exiting 04:26:35 (7076): No heartbeat from core client for 30 sec - exiting 04:26:36 (7076): No heartbeat from core client for 30 sec - exiting 04:26:37 (7076): No heartbeat from core client for 30 sec - exiting 04:26:38 (7076): No heartbeat from core client for 30 sec - exiting 04:26:39 (7076): No heartbeat from core client for 30 sec - exiting 04:26:40 (7076): No heartbeat from core client for 30 sec - exiting 04:26:41 (7076): No heartbeat from core client for 30 sec - exiting 04:26:42 (7076): No heartbeat from core client for 30 sec - exiting 04:26:43 (7076): No heartbeat from core client for 30 sec - exiting 04:26:44 (7076): No heartbeat from core client for 30 sec - exiting 04:26:45 (7076): No heartbeat from core client for 30 sec - exiting 04:26:46 (7076): No heartbeat from core client for 30 sec - exiting 04:26:47 (7076): No heartbeat from core client for 30 sec - exiting 04:26:48 (7076): No heartbeat from core client for 30 sec - exiting 04:26:49 (7076): No heartbeat from core client for 30 sec - exiting 04:26:50 (7076): No heartbeat from core client for 30 sec - exiting 04:26:51 (7076): No heartbeat from core client for 30 sec - exiting 04:26:52 (7076): No heartbeat from core client for 30 sec - exiting 04:26:53 (7076): No heartbeat from core client for 30 sec - exiting 04:26:54 (7076): No heartbeat from core client for 30 sec - exiting 04:26:55 (7076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:26:56 (7076): No heartbeat from core client for 30 sec - exiting 04:28:13 (8920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:42:26 (2456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:21 (2456): No heartbeat from core client for 30 sec - exiting 09:43:22 (2456): No heartbeat from core client for 30 sec - exiting 09:43:24 (2456): No heartbeat from core client for 30 sec - exiting 09:43:25 (2456): No heartbeat from core client for 30 sec - exiting 09:43:26 (2456): No heartbeat from core client for 30 sec - exiting 09:48:05 (8748): No heartbeat from core client for 30 sec - exiting 09:48:06 (8748): No heartbeat from core client for 30 sec - exiting 09:48:07 (8748): No heartbeat from core client for 30 sec - exiting 09:48:08 (8748): No heartbeat from core client for 30 sec - exiting 09:48:09 (8748): No heartbeat from core client for 30 sec - exiting 09:48:10 (8748): No heartbeat from core client for 30 sec - exiting 09:48:11 (8748): No heartbeat from core client for 30 sec - exiting 09:48:12 (8748): No heartbeat from core client for 30 sec - exiting 09:48:13 (8748): No heartbeat from core client for 30 sec - exiting 09:48:14 (8748): No heartbeat from core client for 30 sec - exiting 09:48:15 (8748): No heartbeat from core client for 30 sec - exiting 09:48:16 (8748): No heartbeat from core client for 30 sec - exiting 09:48:17 (8748): No heartbeat from core client for 30 sec - exiting 09:48:18 (8748): No heartbeat from core client for 30 sec - exiting 09:48:19 (8748): No heartbeat from core client for 30 sec - exiting 09:48:21 (8748): No heartbeat from core client for 30 sec - exiting 09:48:22 (8748): No heartbeat from core client for 30 sec - exiting 09:48:23 (8748): No heartbeat from core client for 30 sec - exiting 09:48:24 (8748): No heartbeat from core client for 30 sec - exiting 09:48:25 (8748): No heartbeat from core client for 30 sec - exiting 09:48:26 (8748): No heartbeat from core client for 30 sec - exiting 09:48:27 (8748): No heartbeat from core client for 30 sec - exiting 09:48:28 (8748): No heartbeat from core client for 30 sec - exiting 09:48:29 (8748): No heartbeat from core client for 30 sec - exiting 09:48:30 (8748): No heartbeat from core client for 30 sec - exiting 09:48:31 (8748): No heartbeat from core client for 30 sec - exiting 09:48:32 (8748): No heartbeat from core client for 30 sec - exiting 09:48:33 (8748): No heartbeat from core client for 30 sec - exiting 09:48:34 (8748): No heartbeat from core client for 30 sec - exiting 09:48:35 (8748): No heartbeat from core client for 30 sec - exiting 09:49:06 (8748): No heartbeat from core client for 30 sec - exiting 09:49:07 (8748): No heartbeat from core client for 30 sec - exiting 09:49:08 (8748): No heartbeat from core client for 30 sec - exiting 09:49:09 (8748): No heartbeat from core client for 30 sec - exiting 09:49:10 (8748): No heartbeat from core client for 30 sec - exiting 09:49:11 (8748): No heartbeat from core client for 30 sec - exiting 09:49:12 (8748): No heartbeat from core client for 30 sec - exiting 09:49:13 (8748): No heartbeat from core client for 30 sec - exiting 09:49:14 (8748): No heartbeat from core client for 30 sec - exiting 09:49:15 (8748): No heartbeat from core client for 30 sec - exiting 09:49:16 (8748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:02:55 (5468): No heartbeat from core client for 30 sec - exiting 07:02:56 (5468): No heartbeat from core client for 30 sec - exiting 07:02:57 (5468): No heartbeat from core client for 30 sec - exiting 07:02:58 (5468): No heartbeat from core client for 30 sec - exiting 07:02:59 (5468): No heartbeat from core client for 30 sec - exiting 07:03:00 (5468): No heartbeat from core client for 30 sec - exiting 07:03:01 (5468): No heartbeat from core client for 30 sec - exiting 07:03:02 (5468): No heartbeat from core client for 30 sec - exiting 07:03:03 (5468): No heartbeat from core client for 30 sec - exiting 07:03:04 (5468): No heartbeat from core client for 30 sec - exiting 07:03:05 (5468): No heartbeat from core client for 30 sec - exiting 07:03:06 (5468): No heartbeat from core client for 30 sec - exiting 07:03:07 (5468): No heartbeat from core client for 30 sec - exiting 07:03:08 (5468): No heartbeat from core client for 30 sec - exiting 07:03:09 (5468): No heartbeat from core client for 30 sec - exiting 07:03:10 (5468): No heartbeat from core client for 30 sec - exiting 07:03:11 (5468): No heartbeat from core client for 30 sec - exiting 07:03:12 (5468): No heartbeat from core client for 30 sec - exiting 07:03:13 (5468): No heartbeat from core client for 30 sec - exiting 07:03:14 (5468): No heartbeat from core client for 30 sec - exiting 07:03:15 (5468): No heartbeat from core client for 30 sec - exiting 07:03:16 (5468): No heartbeat from core client for 30 sec - exiting 07:03:17 (5468): No heartbeat from core client for 30 sec - exiting 07:03:18 (5468): No heartbeat from core client for 30 sec - exiting 07:03:19 (5468): No heartbeat from core client for 30 sec - exiting 07:03:20 (5468): No heartbeat from core client for 30 sec - exiting 07:03:21 (5468): No heartbeat from core client for 30 sec - exiting 07:03:22 (5468): No heartbeat from core client for 30 sec - exiting 07:03:23 (5468): No heartbeat from core client for 30 sec - exiting 07:03:24 (5468): No heartbeat from core client for 30 sec - exiting 07:03:25 (5468): No heartbeat from core client for 30 sec - exiting 07:03:26 (5468): No heartbeat from core client for 30 sec - exiting 07:03:27 (5468): No heartbeat from core client for 30 sec - exiting 07:03:28 (5468): No heartbeat from core client for 30 sec - exiting 07:03:29 (5468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:03:30 (5468): No heartbeat from core client for 30 sec - exiting 07:04:49 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:56 (6504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:57 (6504): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:59:31 (4388): No heartbeat from core client for 30 sec - exiting 21:59:33 (4388): No heartbeat from core client for 30 sec - exiting 21:59:34 (4388): No heartbeat from core client for 30 sec - exiting 21:59:35 (4388): No heartbeat from core client for 30 sec - exiting 21:59:36 (4388): No heartbeat from core client for 30 sec - exiting 21:59:37 (4388): No heartbeat from core client for 30 sec - exiting 21:59:38 (4388): No heartbeat from core client for 30 sec - exiting 21:59:39 (4388): No heartbeat from core client for 30 sec - exiting 21:59:40 (4388): No heartbeat from core client for 30 sec - exiting 21:59:41 (4388): No heartbeat from core client for 30 sec - exiting 21:59:42 (4388): No heartbeat from core client for 30 sec - exiting 21:59:43 (4388): No heartbeat from core client for 30 sec - exiting 21:59:44 (4388): No heartbeat from core client for 30 sec - exiting 21:59:45 (4388): No heartbeat from core client for 30 sec - exiting 21:59:46 (4388): No heartbeat from core client for 30 sec - exiting 21:59:47 (4388): No heartbeat from core client for 30 sec - exiting 21:59:48 (4388): No heartbeat from core client for 30 sec - exiting 21:59:49 (4388): No heartbeat from core client for 30 sec - exiting 21:59:50 (4388): No heartbeat from core client for 30 sec - exiting 21:59:51 (4388): No heartbeat from core client for 30 sec - exiting 21:59:52 (4388): No heartbeat from core client for 30 sec - exiting 21:59:53 (4388): No heartbeat from core client for 30 sec - exiting 21:59:54 (4388): No heartbeat from core client for 30 sec - exiting 21:59:55 (4388): No heartbeat from core client for 30 sec - exiting 21:59:56 (4388): No heartbeat from core client for 30 sec - exiting 21:59:57 (4388): No heartbeat from core client for 30 sec - exiting 21:59:58 (4388): No heartbeat from core client for 30 sec - exiting 21:59:59 (4388): No heartbeat from core client for 30 sec - exiting 22:00:00 (4388): No heartbeat from core client for 30 sec - exiting 22:00:01 (4388): No heartbeat from core client for 30 sec - exiting 22:00:02 (4388): No heartbeat from core client for 30 sec - exiting 22:00:03 (4388): No heartbeat from core client for 30 sec - exiting 22:00:04 (4388): No heartbeat from core client for 30 sec - exiting 22:00:05 (4388): No heartbeat from core client for 30 sec - exiting 22:00:06 (4388): No heartbeat from core client for 30 sec - exiting 22:00:07 (4388): No heartbeat from core client for 30 sec - exiting 22:00:08 (4388): No heartbeat from core client for 30 sec - exiting 22:00:09 (4388): No heartbeat from core client for 30 sec - exiting 22:00:10 (4388): No heartbeat from core client for 30 sec - exiting 22:00:11 (4388): No heartbeat from core client for 30 sec - exiting 22:00:12 (4388): No heartbeat from core client for 30 sec - exiting 22:00:13 (4388): No heartbeat from core client for 30 sec - exiting 22:00:14 (4388): No heartbeat from core client for 30 sec - exiting 22:00:15 (4388): No heartbeat from core client for 30 sec - exiting 22:00:16 (4388): No heartbeat from core client for 30 sec - exiting 22:00:17 (4388): No heartbeat from core client for 30 sec - exiting 22:00:18 (4388): No heartbeat from core client for 30 sec - exiting 22:00:19 (4388): No heartbeat from core client for 30 sec - exiting 22:00:20 (4388): No heartbeat from core client for 30 sec - exiting 22:00:21 (4388): No heartbeat from core client for 30 sec - exiting 22:00:22 (4388): No heartbeat from core client for 30 sec - exiting 22:00:23 (4388): No heartbeat from core client for 30 sec - exiting 22:00:24 (4388): No heartbeat from core client for 30 sec - exiting 22:00:25 (4388): No heartbeat from core client for 30 sec - exiting 22:00:26 (4388): No heartbeat from core client for 30 sec - exiting 22:00:27 (4388): No heartbeat from core client for 30 sec - exiting 22:00:28 (4388): No heartbeat from core client for 30 sec - exiting 22:00:29 (4388): No heartbeat from core client for 30 sec - exiting 22:00:30 (4388): No heartbeat from core client for 30 sec - exiting 22:00:31 (4388): No heartbeat from core client for 30 sec - exiting 22:00:32 (4388): No heartbeat from core client for 30 sec - exiting 22:00:33 (4388): No heartbeat from core client for 30 sec - exiting 22:00:34 (4388): No heartbeat from core client for 30 sec - exiting 22:00:35 (4388): No heartbeat from core client for 30 sec - exiting 22:00:36 (4388): No heartbeat from core client for 30 sec - exiting 22:00:37 (4388): No heartbeat from core client for 30 sec - exiting 22:00:38 (4388): No heartbeat from core client for 30 sec - exiting 22:00:39 (4388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:24 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:57:38 (7744): No heartbeat from core client for 30 sec - exiting 05:57:39 (7744): No heartbeat from core client for 30 sec - exiting 05:57:40 (7744): No heartbeat from core client for 30 sec - exiting 05:57:41 (7744): No heartbeat from core client for 30 sec - exiting 05:57:42 (7744): No heartbeat from core client for 30 sec - exiting 05:57:43 (7744): No heartbeat from core client for 30 sec - exiting 05:57:44 (7744): No heartbeat from core client for 30 sec - exiting 05:57:45 (7744): No heartbeat from core client for 30 sec - exiting 05:57:46 (7744): No heartbeat from core client for 30 sec - exiting 05:57:47 (7744): No heartbeat from core client for 30 sec - exiting 05:57:48 (7744): No heartbeat from core client for 30 sec - exiting 05:57:49 (7744): No heartbeat from core client for 30 sec - exiting 05:57:50 (7744): No heartbeat from core client for 30 sec - exiting 05:57:51 (7744): No heartbeat from core client for 30 sec - exiting 05:57:52 (7744): No heartbeat from core client for 30 sec - exiting 05:57:53 (7744): No heartbeat from core client for 30 sec - exiting 05:57:54 (7744): No heartbeat from core client for 30 sec - exiting 05:57:55 (7744): No heartbeat from core client for 30 sec - exiting 05:57:56 (7744): No heartbeat from core client for 30 sec - exiting 05:57:57 (7744): No heartbeat from core client for 30 sec - exiting 05:57:58 (7744): No heartbeat from core client for 30 sec - exiting 05:57:59 (7744): No heartbeat from core client for 30 sec - exiting 05:58:00 (7744): No heartbeat from core client for 30 sec - exiting 05:58:01 (7744): No heartbeat from core client for 30 sec - exiting 05:58:02 (7744): No heartbeat from core client for 30 sec - exiting 05:58:03 (7744): No heartbeat from core client for 30 sec - exiting 05:58:04 (7744): No heartbeat from core client for 30 sec - exiting 05:58:05 (7744): No heartbeat from core client for 30 sec - exiting 05:58:06 (7744): No heartbeat from core client for 30 sec - exiting 05:58:07 (7744): No heartbeat from core client for 30 sec - exiting 05:58:08 (7744): No heartbeat from core client for 30 sec - exiting 05:58:09 (7744): No heartbeat from core client for 30 sec - exiting 05:58:10 (7744): No heartbeat from core client for 30 sec - exiting 05:58:11 (7744): No heartbeat from core client for 30 sec - exiting 05:58:12 (7744): No heartbeat from core client for 30 sec - exiting 05:58:13 (7744): No heartbeat from core client for 30 sec - exiting 05:58:14 (7744): No heartbeat from core client for 30 sec - exiting 05:58:15 (7744): No heartbeat from core client for 30 sec - exiting 05:58:16 (7744): No heartbeat from core client for 30 sec - exiting 05:58:17 (7744): No heartbeat from core client for 30 sec - exiting 05:58:18 (7744): No heartbeat from core client for 30 sec - exiting 05:58:19 (7744): No heartbeat from core client for 30 sec - exiting 05:58:20 (7744): No heartbeat from core client for 30 sec - exiting 05:58:21 (7744): No heartbeat from core client for 30 sec - exiting 05:58:22 (7744): No heartbeat from core client for 30 sec - exiting 05:58:23 (7744): No heartbeat from core client for 30 sec - exiting 05:58:24 (7744): No heartbeat from core client for 30 sec - exiting 05:58:25 (7744): No heartbeat from core client for 30 sec - exiting 05:58:26 (7744): No heartbeat from core client for 30 sec - exiting 05:58:27 (7744): No heartbeat from core client for 30 sec - exiting 05:58:28 (7744): No heartbeat from core client for 30 sec - exiting 05:58:29 (7744): No heartbeat from core client for 30 sec - exiting 05:58:30 (7744): No heartbeat from core client for 30 sec - exiting 05:58:31 (7744): No heartbeat from core client for 30 sec - exiting 05:58:32 (7744): No heartbeat from core client for 30 sec - exiting 05:58:33 (7744): No heartbeat from core client for 30 sec - exiting 05:58:34 (7744): No heartbeat from core client for 30 sec - exiting 05:58:35 (7744): No heartbeat from core client for 30 sec - exiting 05:58:36 (7744): No heartbeat from core client for 30 sec - exiting 05:58:37 (7744): No heartbeat from core client for 30 sec - exiting 05:58:38 (7744): No heartbeat from core client for 30 sec - exiting 05:58:39 (7744): No heartbeat from core client for 30 sec - exiting 05:58:40 (7744): No heartbeat from core client for 30 sec - exiting 05:58:41 (7744): No heartbeat from core client for 30 sec - exiting 05:58:42 (7744): No heartbeat from core client for 30 sec - exiting 05:58:43 (7744): No heartbeat from core client for 30 sec - exiting 05:58:44 (7744): No heartbeat from core client for 30 sec - exiting 05:58:45 (7744): No heartbeat from core client for 30 sec - exiting 05:58:46 (7744): No heartbeat from core client for 30 sec - exiting 05:58:47 (7744): No heartbeat from core client for 30 sec - exiting 05:58:48 (7744): No heartbeat from core client for 30 sec - exiting 05:58:49 (7744): No heartbeat from core client for 30 sec - exiting 05:58:50 (7744): No heartbeat from core client for 30 sec - exiting 05:58:51 (7744): No heartbeat from core client for 30 sec - exiting 05:58:52 (7744): No heartbeat from core client for 30 sec - exiting 05:58:53 (7744): No heartbeat from core client for 30 sec - exiting 05:58:54 (7744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:58:55 (7744): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4532, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4532, iMonCtr=1 Model crash detected, will try to restart... 06:01:41 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1460, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1460, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1460, iMonCtr=1 Model crash detected, will try to restart... 06:03:11 (1460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3904, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Feb 2012 11:01:18	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	388,800	747,605	1.9229
10 Feb 2012 19:57:07	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	362,880	697,568	1.9223
10 Feb 2012 04:59:19	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	336,960	647,265	1.9209
09 Feb 2012 14:06:40	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	311,040	597,311	1.9204
08 Feb 2012 17:17:46	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	285,120	546,419	1.9165
08 Feb 2012 02:36:00	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	259,200	494,909	1.9094
07 Feb 2012 11:54:36	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	233,280	443,047	1.8992
06 Feb 2012 21:49:37	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	207,360	393,577	1.8980
06 Feb 2012 07:38:00	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	181,440	344,291	1.8975
05 Feb 2012 07:07:53	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	155,520	295,599	1.9007
04 Feb 2012 16:28:58	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	129,600	246,280	1.9003
02 Feb 2012 18:49:31	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	103,680	197,416	1.9041
02 Feb 2012 00:33:49	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	77,760	148,529	1.9101
01 Feb 2012 00:10:39	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	51,840	99,680	1.9228
31 Jan 2012 06:09:31	1020546	14023441	hadcm3n_y8vr_1940_40_007742229_4	25,920	49,514	1.9103