Task 15863961

Name	hadcm3n_ze2f_1920_40_008392610_3
Workunit	8543469
Created	25 Jun 2013, 17:22:26 UTC
Sent	25 Jun 2013, 17:33:49 UTC
Report deadline	25 Sep 2013, 1:01:00 UTC
Received	10 Jul 2013, 4:58:44 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1101937
Run time	4 days 19 hours 42 min 40 sec
CPU time	3 days 6 hours 47 min 12 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	1.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 15:00:11 (6556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:00:12 (6556): No heartbeat from core client for 30 sec - exiting 15:01:02 (8060): No heartbeat from core client for 30 sec - exiting 15:01:03 (8060): No heartbeat from core client for 30 sec - exiting 15:01:04 (8060): No heartbeat from core client for 30 sec - exiting 15:01:35 (8060): No heartbeat from core client for 30 sec - exiting 15:01:36 (8060): No heartbeat from core client for 30 sec - exiting 15:01:37 (8060): No heartbeat from core client for 30 sec - exiting 15:01:38 (8060): No heartbeat from core client for 30 sec - exiting 15:01:39 (8060): No heartbeat from core client for 30 sec - exiting 15:01:40 (8060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:53:38 (5996): No heartbeat from core client for 30 sec - exiting 07:53:39 (5996): No heartbeat from core client for 30 sec - exiting 07:53:40 (5996): No heartbeat from core client for 30 sec - exiting 07:53:41 (5996): No heartbeat from core client for 30 sec - exiting 07:53:43 (5996): No heartbeat from core client for 30 sec - exiting 07:53:44 (5996): No heartbeat from core client for 30 sec - exiting 07:53:45 (5996): No heartbeat from core client for 30 sec - exiting 07:53:46 (5996): No heartbeat from core client for 30 sec - exiting 07:53:47 (5996): No heartbeat from core client for 30 sec - exiting 07:53:48 (5996): No heartbeat from core client for 30 sec - exiting 07:53:49 (5996): No heartbeat from core client for 30 sec - exiting 07:53:50 (5996): No heartbeat from core client for 30 sec - exiting 07:53:51 (5996): No heartbeat from core client for 30 sec - exiting 07:53:52 (5996): No heartbeat from core client for 30 sec - exiting 07:53:54 (5996): No heartbeat from core client for 30 sec - exiting 07:53:55 (5996): No heartbeat from core client for 30 sec - exiting 07:53:56 (5996): No heartbeat from core client for 30 sec - exiting 07:53:57 (5996): No heartbeat from core client for 30 sec - exiting 07:53:58 (5996): No heartbeat from core client for 30 sec - exiting 07:53:59 (5996): No heartbeat from core client for 30 sec - exiting 07:54:00 (5996): No heartbeat from core client for 30 sec - exiting 07:54:01 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:54:02 (5996): No heartbeat from core client for 30 sec - exiting 07:55:00 (5380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:55:59 (956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:11:56 (4852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:22:21 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:22:22 (5108): No heartbeat from core client for 30 sec - exiting 03:43:46 (2504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:19 (6768): No heartbeat from core client for 30 sec - exiting 03:44:20 (6768): No heartbeat from core client for 30 sec - exiting 03:44:21 (6768): No heartbeat from core client for 30 sec - exiting 03:44:22 (6768): No heartbeat from core client for 30 sec - exiting 03:44:23 (6768): No heartbeat from core client for 30 sec - exiting 03:44:24 (6768): No heartbeat from core client for 30 sec - exiting 03:44:25 (6768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:10:55 (6980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:12:17 (10040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:12:18 (10040): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:03:04 (3948): No heartbeat from core client for 30 sec - exiting 05:03:05 (3948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:06 (3948): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:22:24 (10228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:23:16 (9148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:10:00 (10244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:11:34 (9160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:33:31 (10424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:14 (12132): No heartbeat from core client for 30 sec - exiting 16:34:15 (12132): No heartbeat from core client for 30 sec - exiting 16:34:16 (12132): No heartbeat from core client for 30 sec - exiting 16:34:17 (12132): No heartbeat from core client for 30 sec - exiting 16:34:18 (12132): No heartbeat from core client for 30 sec - exiting 16:34:19 (12132): No heartbeat from core client for 30 sec - exiting 16:34:20 (12132): No heartbeat from core client for 30 sec - exiting 16:34:21 (12132): No heartbeat from core client for 30 sec - exiting 16:34:22 (12132): No heartbeat from core client for 30 sec - exiting 16:34:23 (12132): No heartbeat from core client for 30 sec - exiting 16:34:24 (12132): No heartbeat from core client for 30 sec - exiting 16:34:25 (12132): No heartbeat from core client for 30 sec - exiting 16:34:26 (12132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:43:50 (13168): No heartbeat from core client for 30 sec - exiting 23:43:51 (13168): No heartbeat from core client for 30 sec - exiting 23:43:52 (13168): No heartbeat from core client for 30 sec - exiting 23:43:53 (13168): No heartbeat from core client for 30 sec - exiting 23:43:54 (13168): No heartbeat from core client for 30 sec - exiting 23:43:55 (13168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:43:56 (13168): No heartbeat from core client for 30 sec - exiting 23:45:58 (12996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:43:14 (10248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:44:05 (9552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:23:37 (11028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:32:16 (3676): No heartbeat from core client for 30 sec - exiting 15:32:17 (3676): No heartbeat from core client for 30 sec - exiting 15:32:18 (3676): No heartbeat from core client for 30 sec - exiting 15:32:19 (3676): No heartbeat from core client for 30 sec - exiting 15:32:20 (3676): No heartbeat from core client for 30 sec - exiting 15:32:21 (3676): No heartbeat from core client for 30 sec - exiting 15:32:22 (3676): No heartbeat from core client for 30 sec - exiting 15:32:23 (3676): No heartbeat from core client for 30 sec - exiting 15:32:25 (3676): No heartbeat from core client for 30 sec - exiting 15:32:26 (3676): No heartbeat from core client for 30 sec - exiting 15:32:27 (3676): No heartbeat from core client for 30 sec - exiting 15:32:28 (3676): No heartbeat from core client for 30 sec - exiting 15:32:29 (3676): No heartbeat from core client for 30 sec - exiting 15:32:30 (3676): No heartbeat from core client for 30 sec - exiting 15:32:31 (3676): No heartbeat from core client for 30 sec - exiting 15:32:32 (3676): No heartbeat from core client for 30 sec - exiting 15:32:33 (3676): No heartbeat from core client for 30 sec - exiting 15:32:34 (3676): No heartbeat from core client for 30 sec - exiting 15:32:35 (3676): No heartbeat from core client for 30 sec - exiting 15:32:37 (3676): No heartbeat from core client for 30 sec - exiting 15:32:38 (3676): No heartbeat from core client for 30 sec - exiting 15:32:39 (3676): No heartbeat from core client for 30 sec - exiting 15:32:40 (3676): No heartbeat from core client for 30 sec - exiting 15:32:41 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:32:42 (3676): No heartbeat from core client for 30 sec - exiting 15:32:43 (3676): No heartbeat from core client for 30 sec - exiting 15:32:44 (3676): No heartbeat from core client for 30 sec - exiting 15:32:45 (3676): No heartbeat from core client for 30 sec - exiting 15:32:46 (3676): No heartbeat from core client for 30 sec - exiting 15:32:48 (3676): No heartbeat from core client for 30 sec - exiting 15:32:49 (3676): No heartbeat from core client for 30 sec - exiting 15:32:50 (3676): No heartbeat from core client for 30 sec - exiting 15:32:51 (3676): No heartbeat from core client for 30 sec - exiting 15:33:55 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:23:37 (9056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:24:14 (11656): No heartbeat from core client for 30 sec - exiting 06:24:15 (11656): No heartbeat from core client for 30 sec - exiting 06:24:16 (11656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:33:45 (10056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:34:22 (12140): No heartbeat from core client for 30 sec - exiting 07:34:23 (12140): No heartbeat from core client for 30 sec - exiting 07:34:24 (12140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:13:54 (12560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:15:03 (10784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:01:05 (11120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:01:48 (11964): No heartbeat from core client for 30 sec - exiting 23:01:49 (11964): No heartbeat from core client for 30 sec - exiting 23:01:50 (11964): No heartbeat from core client for 30 sec - exiting 23:01:51 (11964): No heartbeat from core client for 30 sec - exiting 23:01:52 (11964): No heartbeat from core client for 30 sec - exiting 23:01:53 (11964): No heartbeat from core client for 30 sec - exiting 23:01:54 (11964): No heartbeat from core client for 30 sec - exiting 23:01:55 (11964): No heartbeat from core client for 30 sec - exiting 23:01:56 (11964): No heartbeat from core client for 30 sec - exiting 23:01:57 (11964): No heartbeat from core client for 30 sec - exiting 23:01:58 (11964): No heartbeat from core client for 30 sec - exiting 23:01:59 (11964): No heartbeat from core client for 30 sec - exiting 23:02:00 (11964): No heartbeat from core client for 30 sec - exiting 23:02:01 (11964): No heartbeat from core client for 30 sec - exiting 23:02:02 (11964): No heartbeat from core client for 30 sec - exiting 23:02:03 (11964): No heartbeat from core client for 30 sec - exiting 23:02:04 (11964): No heartbeat from core client for 30 sec - exiting 23:02:05 (11964): No heartbeat from core client for 30 sec - exiting 23:02:06 (11964): No heartbeat from core client for 30 sec - exiting 23:02:07 (11964): No heartbeat from core client for 30 sec - exiting 23:02:08 (11964): No heartbeat from core client for 30 sec - exiting 23:02:09 (11964): No heartbeat from core client for 30 sec - exiting 23:02:10 (11964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:02:11 (11964): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 23:06:06 (9252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:43 (13864): No heartbeat from core client for 30 sec - exiting 23:06:44 (13864): No heartbeat from core client for 30 sec - exiting 23:06:45 (13864): No heartbeat from core client for 30 sec - exiting 23:06:46 (13864): No heartbeat from core client for 30 sec - exiting 23:06:47 (13864): No heartbeat from core client for 30 sec - exiting 23:06:48 (13864): No heartbeat from core client for 30 sec - exiting 23:06:49 (13864): No heartbeat from core client for 30 sec - exiting 23:06:50 (13864): No heartbeat from core client for 30 sec - exiting 23:06:51 (13864): No heartbeat from core client for 30 sec - exiting 23:06:52 (13864): No heartbeat from core client for 30 sec - exiting 23:06:53 (13864): No heartbeat from core client for 30 sec - exiting 23:06:54 (13864): No heartbeat from core client for 30 sec - exiting 23:06:55 (13864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:08:05 (13988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:16:06 (14056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:26:29 (9388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:23 (8568): No heartbeat from core client for 30 sec - exiting 03:27:24 (8568): No heartbeat from core client for 30 sec - exiting 03:27:25 (8568): No heartbeat from core client for 30 sec - exiting 03:27:27 (8568): No heartbeat from core client for 30 sec - exiting 03:27:28 (8568): No heartbeat from core client for 30 sec - exiting 03:27:29 (8568): No heartbeat from core client for 30 sec - exiting 03:27:30 (8568): No heartbeat from core client for 30 sec - exiting 03:27:31 (8568): No heartbeat from core client for 30 sec - exiting 03:27:32 (8568): No heartbeat from core client for 30 sec - exiting 03:27:33 (8568): No heartbeat from core client for 30 sec - exiting 03:27:34 (8568): No heartbeat from core client for 30 sec - exiting 03:27:35 (8568): No heartbeat from core client for 30 sec - exiting 03:27:36 (8568): No heartbeat from core client for 30 sec - exiting 03:27:37 (8568): No heartbeat from core client for 30 sec - exiting 03:27:38 (8568): No heartbeat from core client for 30 sec - exiting 03:27:39 (8568): No heartbeat from core client for 30 sec - exiting 03:27:40 (8568): No heartbeat from core client for 30 sec - exiting 03:27:41 (8568): No heartbeat from core client for 30 sec - exiting 03:27:42 (8568): No heartbeat from core client for 30 sec - exiting 03:27:43 (8568): No heartbeat from core client for 30 sec - exiting 03:27:44 (8568): No heartbeat from core client for 30 sec - exiting 03:27:45 (8568): No heartbeat from core client for 30 sec - exiting 03:27:46 (8568): No heartbeat from core client for 30 sec - exiting 03:27:47 (8568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:48 (8568): No heartbeat from core client for 30 sec - exiting 03:27:49 (8568): No heartbeat from core client for 30 sec - exiting 03:27:50 (8568): No heartbeat from core client for 30 sec - exiting 03:27:51 (8568): No heartbeat from core client for 30 sec - exiting 03:27:52 (8568): No heartbeat from core client for 30 sec - exiting 03:27:53 (8568): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:19:38 (7260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:24 (12260): No heartbeat from core client for 30 sec - exiting 05:20:25 (12260): No heartbeat from core client for 30 sec - exiting 05:20:26 (12260): No heartbeat from core client for 30 sec - exiting 05:20:27 (12260): No heartbeat from core client for 30 sec - exiting 05:20:28 (12260): No heartbeat from core client for 30 sec - exiting 05:20:29 (12260): No heartbeat from core client for 30 sec - exiting 05:20:30 (12260): No heartbeat from core client for 30 sec - exiting 05:20:31 (12260): No heartbeat from core client for 30 sec - exiting 05:20:32 (12260): No heartbeat from core client for 30 sec - exiting 05:20:33 (12260): No heartbeat from core client for 30 sec - exiting 05:20:34 (12260): No heartbeat from core client for 30 sec - exiting 05:20:35 (12260): No heartbeat from core client for 30 sec - exiting 05:20:36 (12260): No heartbeat from core client for 30 sec - exiting 05:20:37 (12260): No heartbeat from core client for 30 sec - exiting 05:20:38 (12260): No heartbeat from core client for 30 sec - exiting 05:20:39 (12260): No heartbeat from core client for 30 sec - exiting 05:20:40 (12260): No heartbeat from core client for 30 sec - exiting 05:20:41 (12260): No heartbeat from core client for 30 sec - exiting 05:20:42 (12260): No heartbeat from core client for 30 sec - exiting 05:20:43 (12260): No heartbeat from core client for 30 sec - exiting 05:20:44 (12260): No heartbeat from core client for 30 sec - exiting 05:20:45 (12260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:49 (2272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:02:06 (7436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:42 (10176): No heartbeat from core client for 30 sec - exiting 08:02:43 (10176): No heartbeat from core client for 30 sec - exiting 08:02:44 (10176): No heartbeat from core client for 30 sec - exiting 08:02:45 (10176): No heartbeat from core client for 30 sec - exiting 08:02:46 (10176): No heartbeat from core client for 30 sec - exiting 08:02:47 (10176): No heartbeat from core client for 30 sec - exiting 08:02:48 (10176): No heartbeat from core client for 30 sec - exiting 08:02:49 (10176): No heartbeat from core client for 30 sec - exiting 08:02:50 (10176): No heartbeat from core client for 30 sec - exiting 08:02:52 (10176): No heartbeat from core client for 30 sec - exiting 08:02:53 (10176): No heartbeat from core client for 30 sec - exiting 08:02:54 (10176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:03:57 (11108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:05:05 (3536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:39 (11524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:40 (11524): No heartbeat from core client for 30 sec - exiting 08:06:39 (13424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:12:56 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:13:30 (7892): No heartbeat from core client for 30 sec - exiting 08:13:31 (7892): No heartbeat from core client for 30 sec - exiting 08:13:32 (7892): No heartbeat from core client for 30 sec - exiting 08:13:33 (7892): No heartbeat from core client for 30 sec - exiting 08:13:34 (7892): No heartbeat from core client for 30 sec - exiting 08:13:35 (7892): No heartbeat from core client for 30 sec - exiting 08:13:36 (7892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:33:28 (9900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:11 (11388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:53:47 (7668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:34:47 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:35:33 (10772): No heartbeat from core client for 30 sec - exiting 12:35:34 (10772): No heartbeat from core client for 30 sec - exiting 12:35:35 (10772): No heartbeat from core client for 30 sec - exiting 12:35:36 (10772): No heartbeat from core client for 30 sec - exiting 12:35:37 (10772): No heartbeat from core client for 30 sec - exiting 12:35:38 (10772): No heartbeat from core client for 30 sec - exiting 12:35:39 (10772): No heartbeat from core client for 30 sec - exiting 12:35:40 (10772): No heartbeat from core client for 30 sec - exiting 12:35:41 (10772): No heartbeat from core client for 30 sec - exiting 12:35:42 (10772): No heartbeat from core client for 30 sec - exiting 12:35:43 (10772): No heartbeat from core client for 30 sec - exiting 12:35:44 (10772): No heartbeat from core client for 30 sec - exiting 12:35:45 (10772): No heartbeat from core client for 30 sec - exiting 12:35:46 (10772): No heartbeat from core client for 30 sec - exiting 12:35:47 (10772): No heartbeat from core client for 30 sec - exiting 12:35:48 (10772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:55:07 (7948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:55:56 (1172): No heartbeat from core client for 30 sec - exiting 15:55:57 (1172): No heartbeat from core client for 30 sec - exiting 15:55:58 (1172): No heartbeat from core client for 30 sec - exiting 15:55:59 (1172): No heartbeat from core client for 30 sec - exiting 15:56:00 (1172): No heartbeat from core client for 30 sec - exiting 15:56:01 (1172): No heartbeat from core client for 30 sec - exiting 15:56:02 (1172): No heartbeat from core client for 30 sec - exiting 15:56:03 (1172): No heartbeat from core client for 30 sec - exiting 15:56:04 (1172): No heartbeat from core client for 30 sec - exiting 15:56:05 (1172): No heartbeat from core client for 30 sec - exiting 15:56:06 (1172): No heartbeat from core client for 30 sec - exiting 15:56:07 (1172): No heartbeat from core client for 30 sec - exiting 15:56:08 (1172): No heartbeat from core client for 30 sec - exiting 15:56:09 (1172): No heartbeat from core client for 30 sec - exiting 15:56:10 (1172): No heartbeat from core client for 30 sec - exiting 15:56:11 (1172): No heartbeat from core client for 30 sec - exiting 15:56:12 (1172): No heartbeat from core client for 30 sec - exiting 15:56:13 (1172): No heartbeat from core client for 30 sec - exiting 15:56:14 (1172): No heartbeat from core client for 30 sec - exiting 15:56:15 (1172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:00:08 (11488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:43 (13428): No heartbeat from core client for 30 sec - exiting 16:00:44 (13428): No heartbeat from core client for 30 sec - exiting 16:00:45 (13428): No heartbeat from core client for 30 sec - exiting 16:00:46 (13428): No heartbeat from core client for 30 sec - exiting 16:00:47 (13428): No heartbeat from core client for 30 sec - exiting 16:00:48 (13428): No heartbeat from core client for 30 sec - exiting 16:00:49 (13428): No heartbeat from core client for 30 sec - exiting 16:00:50 (13428): No heartbeat from core client for 30 sec - exiting 16:00:51 (13428): No heartbeat from core client for 30 sec - exiting 16:00:52 (13428): No heartbeat from core client for 30 sec - exiting 16:00:53 (13428): No heartbeat from core client for 30 sec - exiting 16:00:54 (13428): No heartbeat from core client for 30 sec - exiting 16:00:55 (13428): No heartbeat from core client for 30 sec - exiting 16:00:56 (13428): No heartbeat from core client for 30 sec - exiting 16:00:57 (13428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:45:12 (10076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:58 (4896): No heartbeat from core client for 30 sec - exiting 16:46:59 (4896): No heartbeat from core client for 30 sec - exiting 16:47:00 (4896): No heartbeat from core client for 30 sec - exiting 16:47:01 (4896): No heartbeat from core client for 30 sec - exiting 16:47:02 (4896): No heartbeat from core client for 30 sec - exiting 16:47:03 (4896): No heartbeat from core client for 30 sec - exiting 16:47:04 (4896): No heartbeat from core client for 30 sec - exiting 16:47:05 (4896): No heartbeat from core client for 30 sec - exiting 16:47:07 (4896): No heartbeat from core client for 30 sec - exiting 16:47:08 (4896): No heartbeat from core client for 30 sec - exiting 16:47:09 (4896): No heartbeat from core client for 30 sec - exiting 16:47:10 (4896): No heartbeat from core client for 30 sec - exiting 16:47:11 (4896): No heartbeat from core client for 30 sec - exiting 16:47:12 (4896): No heartbeat from core client for 30 sec - exiting 16:47:13 (4896): No heartbeat from core client for 30 sec - exiting 16:47:14 (4896): No heartbeat from core client for 30 sec - exiting 16:47:15 (4896): No heartbeat from core client for 30 sec - exiting 16:47:16 (4896): No heartbeat from core client for 30 sec - exiting 16:47:17 (4896): No heartbeat from core client for 30 sec - exiting 16:47:18 (4896): No heartbeat from core client for 30 sec - exiting 16:47:19 (4896): No heartbeat from core client for 30 sec - exiting 16:47:20 (4896): No heartbeat from core client for 30 sec - exiting 16:47:21 (4896): No heartbeat from core client for 30 sec - exiting 16:47:22 (4896): No heartbeat from core client for 30 sec - exiting 16:47:23 (4896): No heartbeat from core client for 30 sec - exiting 16:47:24 (4896): No heartbeat from core client for 30 sec - exiting 16:47:25 (4896): No heartbeat from core client for 30 sec - exiting 16:47:26 (4896): No heartbeat from core client for 30 sec - exiting 16:47:27 (4896): No heartbeat from core client for 30 sec - exiting 16:47:28 (4896): No heartbeat from core client for 30 sec - exiting 16:47:29 (4896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:47:30 (4896): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 16:50:13 (11360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:02 (12320): No heartbeat from core client for 30 sec - exiting 16:51:03 (12320): No heartbeat from core client for 30 sec - exiting 16:51:04 (12320): No heartbeat from core client for 30 sec - exiting 16:51:05 (12320): No heartbeat from core client for 30 sec - exiting 16:51:06 (12320): No heartbeat from core client for 30 sec - exiting 16:51:07 (12320): No heartbeat from core client for 30 sec - exiting 16:51:08 (12320): No heartbeat from core client for 30 sec - exiting 16:51:09 (12320): No heartbeat from core client for 30 sec - exiting 16:51:10 (12320): No heartbeat from core client for 30 sec - exiting 16:51:11 (12320): No heartbeat from core client for 30 sec - exiting 16:51:12 (12320): No heartbeat from core client for 30 sec - exiting 16:51:13 (12320): No heartbeat from core client for 30 sec - exiting 16:51:14 (12320): No heartbeat from core client for 30 sec - exiting 16:51:15 (12320): No heartbeat from core client for 30 sec - exiting 16:51:16 (12320): No heartbeat from core client for 30 sec - exiting 16:51:17 (12320): No heartbeat from core client for 30 sec - exiting 16:51:18 (12320): No heartbeat from core client for 30 sec - exiting 16:51:19 (12320): No heartbeat from core client for 30 sec - exiting 16:51:20 (12320): No heartbeat from core client for 30 sec - exiting 16:51:21 (12320): No heartbeat from core client for 30 sec - exiting 16:51:22 (12320): No heartbeat from core client for 30 sec - exiting 16:51:23 (12320): No heartbeat from core client for 30 sec - exiting 16:51:24 (12320): No heartbeat from core client for 30 sec - exiting 16:51:25 (12320): No heartbeat from core client for 30 sec - exiting 16:51:26 (12320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jul 2013 22:46:58	1101937	15863961	hadcm3n_ze2f_1920_40_008392610_3	51,840	235,032	4.5338
06 Jul 2013 05:21:23	1101937	15863961	hadcm3n_ze2f_1920_40_008392610_3	25,920	117,713	4.5414