Task 15857253

Name	hadcm3n_n7fw_1920_40_008334551_1
Workunit	8485412
Created	22 Jun 2013, 6:58:41 UTC
Sent	23 Jun 2013, 4:19:26 UTC
Report deadline	22 Sep 2013, 11:46:37 UTC
Received	26 Jun 2013, 12:38:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	330080
Run time	13 hours 0 min 32 sec
CPU time	13 hours 0 min 32 sec
Validate state	Invalid
Credit	311.04
Device peak FLOPS	1.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.8.16</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 16:02:40 (3732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:27 (8256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:37 (5972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:47 (8704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:56 (10400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:58 (7388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:43 (9696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:05:27 (12024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:03 (8224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:28:08 (13128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:37:01 (6936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:09 (9760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:15 (9396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:16:58 (7840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:24:42 (9252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:39:49 (980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:22 (12552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 18:52:50 (7664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:01:04 (484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:04 (13056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:35 (9380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:21:46 (9708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:46 (1932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:38:54 (9656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:49:02 (11968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:47 (7564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:01:04 (8756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:20 (9308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:16:38 (2336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:44 (5296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:50 (9012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:30:01 (7096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:42:18 (12344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:44:14 (7648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:42 (7624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:55:07 (8668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:59:24 (9752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:37 (8472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:30 (12948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:33 (12948): No heartbeat from core client for 30 sec - exiting 21:08:34 (12948): No heartbeat from core client for 30 sec - exiting 21:08:35 (12948): No heartbeat from core client for 30 sec - exiting 21:10:40 (8636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:33 (10360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:35 (10360): No heartbeat from core client for 30 sec - exiting 21:18:36 (10360): No heartbeat from core client for 30 sec - exiting 21:20:19 (11300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:20 (11300): No heartbeat from core client for 30 sec - exiting 21:20:21 (11300): No heartbeat from core client for 30 sec - exiting 21:20:23 (11300): No heartbeat from core client for 30 sec - exiting 21:20:24 (11300): No heartbeat from core client for 30 sec - exiting 21:20:25 (11300): No heartbeat from core client for 30 sec - exiting 21:20:26 (11300): No heartbeat from core client for 30 sec - exiting 21:28:33 (9604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:28:34 (9604): No heartbeat from core client for 30 sec - exiting 21:30:07 (3128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:36:36 (13772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:56 (8260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:58 (8260): No heartbeat from core client for 30 sec - exiting 21:45:24 (13464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:49:51 (10400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:49:52 (10400): No heartbeat from core client for 30 sec - exiting 21:55:30 (4516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:55:37 (4516): No heartbeat from core client for 30 sec - exiting 21:55:38 (4516): No heartbeat from core client for 30 sec - exiting 21:55:40 (4516): No heartbeat from core client for 30 sec - exiting 21:55:41 (4516): No heartbeat from core client for 30 sec - exiting 21:55:42 (4516): No heartbeat from core client for 30 sec - exiting 21:55:43 (4516): No heartbeat from core client for 30 sec - exiting 21:58:40 (10964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:58:41 (10964): No heartbeat from core client for 30 sec - exiting 21:58:42 (10964): No heartbeat from core client for 30 sec - exiting 22:05:31 (3924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:35 (3924): No heartbeat from core client for 30 sec - exiting 22:05:36 (3924): No heartbeat from core client for 30 sec - exiting 22:05:37 (3924): No heartbeat from core client for 30 sec - exiting 22:05:38 (3924): No heartbeat from core client for 30 sec - exiting 22:05:39 (3924): No heartbeat from core client for 30 sec - exiting 22:09:53 (14084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:16 (6724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:19 (6724): No heartbeat from core client for 30 sec - exiting 22:20:20 (6724): No heartbeat from core client for 30 sec - exiting 22:20:21 (6724): No heartbeat from core client for 30 sec - exiting 22:20:22 (6724): No heartbeat from core client for 30 sec - exiting 22:25:17 (6420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:30:18 (11848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:30:20 (11848): No heartbeat from core client for 30 sec - exiting 22:30:21 (11848): No heartbeat from core client for 30 sec - exiting 22:33:05 (12476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:40:18 (11620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:40:20 (11620): No heartbeat from core client for 30 sec - exiting 22:45:20 (9608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:56:35 (12868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:56:37 (12868): No heartbeat from core client for 30 sec - exiting 22:56:38 (12868): No heartbeat from core client for 30 sec - exiting 22:56:39 (12868): No heartbeat from core client for 30 sec - exiting 22:56:41 (12868): No heartbeat from core client for 30 sec - exiting 22:56:42 (12868): No heartbeat from core client for 30 sec - exiting 22:58:37 (7000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:01:35 (11984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:03:50 (7456): No heartbeat from core client for 30 sec - exiting 23:03:51 (7456): No heartbeat from core client for 30 sec - exiting 23:03:52 (7456): No heartbeat from core client for 30 sec - exiting 23:03:53 (7456): No heartbeat from core client for 30 sec - exiting 23:03:54 (7456): No heartbeat from core client for 30 sec - exiting 23:03:55 (7456): No heartbeat from core client for 30 sec - exiting 23:03:56 (7456): No heartbeat from core client for 30 sec - exiting 23:03:57 (7456): No heartbeat from core client for 30 sec - exiting 23:03:58 (7456): No heartbeat from core client for 30 sec - exiting 23:03:59 (7456): No heartbeat from core client for 30 sec - exiting 23:04:00 (7456): No heartbeat from core client for 30 sec - exiting 23:04:02 (7456): No heartbeat from core client for 30 sec - exiting 23:04:03 (7456): No heartbeat from core client for 30 sec - exiting 23:04:04 (7456): No heartbeat from core client for 30 sec - exiting 23:04:05 (7456): No heartbeat from core client for 30 sec - exiting 23:04:06 (7456): No heartbeat from core client for 30 sec - exiting 23:04:07 (7456): No heartbeat from core client for 30 sec - exiting 23:04:08 (7456): No heartbeat from core client for 30 sec - exiting 23:04:09 (7456): No heartbeat from core client for 30 sec - exiting 23:04:10 (7456): No heartbeat from core client for 30 sec - exiting 23:04:11 (7456): No heartbeat from core client for 30 sec - exiting 23:04:12 (7456): No heartbeat from core client for 30 sec - exiting 23:04:14 (7456): No heartbeat from core client for 30 sec - exiting 23:04:15 (7456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:35 (6944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:36 (6944): No heartbeat from core client for 30 sec - exiting 23:11:36 (13024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:11:37 (13024): No heartbeat from core client for 30 sec - exiting 23:13:03 (14160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:16:37 (2620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:16:40 (2620): No heartbeat from core client for 30 sec - exiting 23:16:41 (2620): No heartbeat from core client for 30 sec - exiting 23:16:42 (2620): No heartbeat from core client for 30 sec - exiting 23:16:43 (2620): No heartbeat from core client for 30 sec - exiting 23:16:45 (2620): No heartbeat from core client for 30 sec - exiting 23:21:37 (12276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:38 (12276): No heartbeat from core client for 30 sec - exiting 23:21:39 (12276): No heartbeat from core client for 30 sec - exiting 23:21:40 (12276): No heartbeat from core client for 30 sec - exiting 23:23:02 (5756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:32:35 (8796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:32:39 (8796): No heartbeat from core client for 30 sec - exiting 23:32:40 (8796): No heartbeat from core client for 30 sec - exiting 23:35:28 (9724): No heartbeat from core client for 30 sec - exiting 23:35:29 (9724): No heartbeat from core client for 30 sec - exiting 23:35:30 (9724): No heartbeat from core client for 30 sec - exiting 23:35:31 (9724): No heartbeat from core client for 30 sec - exiting 23:35:32 (9724): No heartbeat from core client for 30 sec - exiting 23:35:33 (9724): No heartbeat from core client for 30 sec - exiting 23:35:34 (9724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:35:35 (9724): No heartbeat from core client for 30 sec - exiting 23:35:36 (9724): No heartbeat from core client for 30 sec - exiting 23:35:38 (9724): No heartbeat from core client for 30 sec - exiting 23:35:39 (9724): No heartbeat from core client for 30 sec - exiting 23:35:40 (9724): No heartbeat from core client for 30 sec - exiting 23:35:41 (9724): No heartbeat from core client for 30 sec - exiting 23:35:42 (9724): No heartbeat from core client for 30 sec - exiting 23:35:43 (9724): No heartbeat from core client for 30 sec - exiting 23:35:44 (9724): No heartbeat from core client for 30 sec - exiting 23:35:45 (9724): No heartbeat from core client for 30 sec - exiting 23:42:35 (8220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:42:36 (8220): No heartbeat from core client for 30 sec - exiting 23:42:38 (8220): No heartbeat from core client for 30 sec - exiting 23:42:39 (8220): No heartbeat from core client for 30 sec - exiting 23:44:17 (6692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:30 (4656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:37 (4656): No heartbeat from core client for 30 sec - exiting 23:55:38 (4656): No heartbeat from core client for 30 sec - exiting 23:55:39 (4656): No heartbeat from core client for 30 sec - exiting 23:55:41 (4656): No heartbeat from core client for 30 sec - exiting 23:55:42 (4656): No heartbeat from core client for 30 sec - exiting 23:58:12 (9096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:58:13 (9096): No heartbeat from core client for 30 sec - exiting 23:58:14 (9096): No heartbeat from core client for 30 sec - exiting 23:58:15 (9096): No heartbeat from core client for 30 sec - exiting 23:58:17 (9096): No heartbeat from core client for 30 sec - exiting 23:58:18 (9096): No heartbeat from core client for 30 sec - exiting 23:58:19 (9096): No heartbeat from core client for 30 sec - exiting 23:58:20 (9096): No heartbeat from core client for 30 sec - exiting 23:58:21 (9096): No heartbeat from core client for 30 sec - exiting 23:58:22 (9096): No heartbeat from core client for 30 sec - exiting 00:11:34 (12072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:11:35 (12072): No heartbeat from core client for 30 sec - exiting 00:11:37 (12072): No heartbeat from core client for 30 sec - exiting 00:11:38 (12072): No heartbeat from core client for 30 sec - exiting 00:11:39 (12072): No heartbeat from core client for 30 sec - exiting 00:11:40 (12072): No heartbeat from core client for 30 sec - exiting 00:11:41 (12072): No heartbeat from core client for 30 sec - exiting 00:11:42 (12072): No heartbeat from core client for 30 sec - exiting 00:11:43 (12072): No heartbeat from core client for 30 sec - exiting 00:11:44 (12072): No heartbeat from core client for 30 sec - exiting 00:11:45 (12072): No heartbeat from core client for 30 sec - exiting 00:15:00 (1924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:20:57 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:21:01 (5728): No heartbeat from core client for 30 sec - exiting 00:21:02 (5728): No heartbeat from core client for 30 sec - exiting 00:21:03 (5728): No heartbeat from core client for 30 sec - exiting 00:21:04 (5728): No heartbeat from core client for 30 sec - exiting 00:21:06 (5728): No heartbeat from core client for 30 sec - exiting 00:26:37 (8096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:26:38 (8096): No heartbeat from core client for 30 sec - exiting 00:28:08 (8544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:31:38 (9772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:31:39 (9772): No heartbeat from core client for 30 sec - exiting 00:31:41 (9772): No heartbeat from core client for 30 sec - exiting 00:34:50 (8472): No heartbeat from core client for 30 sec - exiting 00:34:51 (8472): No heartbeat from core client for 30 sec - exiting 00:34:52 (8472): No heartbeat from core client for 30 sec - exiting 00:34:53 (8472): No heartbeat from core client for 30 sec - exiting 00:34:54 (8472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:34:56 (8472): No heartbeat from core client for 30 sec - exiting 00:34:57 (8472): No heartbeat from core client for 30 sec - exiting 00:34:58 (8472): No heartbeat from core client for 30 sec - exiting 00:34:59 (8472): No heartbeat from core client for 30 sec - exiting 00:35:00 (8472): No heartbeat from core client for 30 sec - exiting 00:35:01 (8472): No heartbeat from core client for 30 sec - exiting 00:35:02 (8472): No heartbeat from core client for 30 sec - exiting 00:35:03 (8472): No heartbeat from core client for 30 sec - exiting 00:35:04 (8472): No heartbeat from core client for 30 sec - exiting 00:35:05 (8472): No heartbeat from core client for 30 sec - exiting 00:36:51 (8028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:43:11 (13812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:43:12 (13812): No heartbeat from core client for 30 sec - exiting 00:43:13 (13812): No heartbeat from core client for 30 sec - exiting 00:47:11 (12000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:14 (12000): No heartbeat from core client for 30 sec - exiting 00:47:15 (12000): No heartbeat from core client for 30 sec - exiting 00:47:16 (12000): No heartbeat from core client for 30 sec - exiting 00:47:17 (12000): No heartbeat from core client for 30 sec - exiting 00:52:10 (6988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:52:11 (6988): No heartbeat from core client for 30 sec - exiting 00:54:03 (7348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:57:35 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:57:36 (4472): No heartbeat from core client for 30 sec - exiting 00:57:38 (4472): No heartbeat from core client for 30 sec - exiting 01:02:11 (11584): No heartbeat from core client for 30 sec - exiting 01:02:22 (11584): No heartbeat from core client for 30 sec - exiting 01:02:23 (11584): No heartbeat from core client for 30 sec - exiting 01:02:25 (11584): No heartbeat from core client for 30 sec - exiting 01:02:26 (11584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:02:27 (11584): No heartbeat from core client for 30 sec - exiting 01:02:28 (11584): No heartbeat from core client for 30 sec - exiting 01:02:29 (11584): No heartbeat from core client for 30 sec - exiting 01:02:30 (11584): No heartbeat from core client for 30 sec - exiting 01:02:31 (11584): No heartbeat from core client for 30 sec - exiting 01:02:32 (11584): No heartbeat from core client for 30 sec - exiting 01:02:33 (11584): No heartbeat from core client for 30 sec - exiting 01:02:34 (11584): No heartbeat from core client for 30 sec - exiting 01:02:36 (11584): No heartbeat from core client for 30 sec - exiting 01:02:37 (11584): No heartbeat from core client for 30 sec - exiting 01:02:38 (11584): No heartbeat from core client for 30 sec - exiting 01:02:39 (11584): No heartbeat from core client for 30 sec - exiting 01:02:40 (11584): No heartbeat from core client for 30 sec - exiting 01:02:41 (11584): No heartbeat from core client for 30 sec - exiting 01:02:42 (11584): No heartbeat from core client for 30 sec - exiting 01:02:43 (11584): No heartbeat from core client for 30 sec - exiting 01:06:17 (12140): No heartbeat from core client for 30 sec - exiting 01:06:18 (12140): No heartbeat from core client for 30 sec - exiting 01:06:19 (12140): No heartbeat from core client for 30 sec - exiting 01:06:20 (12140): No heartbeat from core client for 30 sec - exiting 01:06:21 (12140): No heartbeat from core client for 30 sec - exiting 01:06:22 (12140): No heartbeat from core client for 30 sec - exiting 01:06:23 (12140): No heartbeat from core client for 30 sec - exiting 01:06:24 (12140): No heartbeat from core client for 30 sec - exiting 01:06:26 (12140): No heartbeat from core client for 30 sec - exiting 01:06:27 (12140): No heartbeat from core client for 30 sec - exiting 01:06:28 (12140): No heartbeat from core client for 30 sec - exiting 01:06:29 (12140): No heartbeat from core client for 30 sec - exiting 01:06:30 (12140): No heartbeat from core client for 30 sec - exiting 01:06:31 (12140): No heartbeat from core client for 30 sec - exiting 01:06:32 (12140): No heartbeat from core client for 30 sec - exiting 01:06:33 (12140): No heartbeat from core client for 30 sec - exiting 01:06:34 (12140): No heartbeat from core client for 30 sec - exiting 01:06:35 (12140): No heartbeat from core client for 30 sec - exiting 01:06:36 (12140): No heartbeat from core client for 30 sec - exiting 01:06:38 (12140): No heartbeat from core client for 30 sec - exiting 01:06:39 (12140): No heartbeat from core client for 30 sec - exiting 01:06:40 (12140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:06:41 (12140): No heartbeat from core client for 30 sec - exiting 01:06:42 (12140): No heartbeat from core client for 30 sec - exiting 01:06:43 (12140): No heartbeat from core client for 30 sec - exiting 01:06:44 (12140): No heartbeat from core client for 30 sec - exiting 01:06:45 (12140): No heartbeat from core client for 30 sec - exiting 01:06:46 (12140): No heartbeat from core client for 30 sec - exiting 03:09:14 (9600): No heartbeat from core client for 30 sec - exiting 03:09:15 (9600): No heartbeat from core client for 30 sec - exiting 03:09:16 (9600): No heartbeat from core client for 30 sec - exiting 03:09:17 (9600): No heartbeat from core client for 30 sec - exiting 03:09:18 (9600): No heartbeat from core client for 30 sec - exiting 03:09:19 (9600): No heartbeat from core client for 30 sec - exiting 03:09:20 (9600): No heartbeat from core client for 30 sec - exiting 03:09:22 (9600): No heartbeat from core client for 30 sec - exiting 03:09:23 (9600): No heartbeat from core client for 30 sec - exiting 03:09:24 (9600): No heartbeat from core client for 30 sec - exiting 03:09:25 (9600): No heartbeat from core client for 30 sec - exiting 03:09:26 (9600): No heartbeat from core client for 30 sec - exiting 03:09:27 (9600): No heartbeat from core client for 30 sec - exiting 03:09:28 (9600): No heartbeat from core client for 30 sec - exiting 03:09:29 (9600): No heartbeat from core client for 30 sec - exiting 03:09:30 (9600): No heartbeat from core client for 30 sec - exiting 03:09:32 (9600): No heartbeat from core client for 30 sec - exiting 03:09:33 (9600): No heartbeat from core client for 30 sec - exiting 03:09:34 (9600): No heartbeat from core client for 30 sec - exiting 03:09:35 (9600): No heartbeat from core client for 30 sec - exiting 03:09:36 (9600): No heartbeat from core client for 30 sec - exiting 03:09:37 (9600): No heartbeat from core client for 30 sec - exiting 03:09:38 (9600): No heartbeat from core client for 30 sec - exiting 03:09:39 (9600): No heartbeat from core client for 30 sec - exiting 03:09:40 (9600): No heartbeat from core client for 30 sec - exiting 03:09:41 (9600): No heartbeat from core client for 30 sec - exiting 03:09:42 (9600): No heartbeat from core client for 30 sec - exiting 03:09:44 (9600): No heartbeat from core client for 30 sec - exiting 03:09:45 (9600): No heartbeat from core client for 30 sec - exiting 03:09:46 (9600): No heartbeat from core client for 30 sec - exiting 03:09:47 (9600): No heartbeat from core client for 30 sec - exiting 03:09:48 (9600): No heartbeat from core client for 30 sec - exiting 03:09:49 (9600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:09:50 (9600): No heartbeat from core client for 30 sec - exiting 03:09:51 (9600): No heartbeat from core client for 30 sec - exiting 03:09:52 (9600): No heartbeat from core client for 30 sec - exiting 03:09:53 (9600): No heartbeat from core client for 30 sec - exiting 03:09:54 (9600): No heartbeat from core client for 30 sec - exiting 03:09:56 (9600): No heartbeat from core client for 30 sec - exiting 03:09:57 (9600): No heartbeat from core client for 30 sec - exiting 03:09:58 (9600): No heartbeat from core client for 30 sec - exiting 03:09:59 (9600): No heartbeat from core client for 30 sec - exiting 03:10:00 (9600): No heartbeat from core client for 30 sec - exiting 03:12:06 (4540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:20:31 (4688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:28:45 (6548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:29:35 (8028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:31:50 (9488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:42:07 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:45:05 (14232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:16:30 (12044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:47:46 (2332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:57:31 (9396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:19:01 (8060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:05 (13852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:51:01 (6664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:22:56 (7680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:25:48 (11476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:29:13 (13380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:48 (7644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:02:28 (10852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:19:08 (6164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:35:15 (5596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:37:16 (10844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:39:45 (11524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:42:42 (9336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:45:37 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:04:07 (14084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:14:33 (11352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:46:30 (11612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:07 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:17:45 (8180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:18:26 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:21:34 (12444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:51:33 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:43 (10348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:48:23 (8264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:48:25 (8264): No heartbeat from core client for 30 sec - exiting 10:53:33 (12456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:56:59 (9256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:30:11 (5964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:39:54 (10896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:03:00 (9224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:17:18 (10556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:36:33 (13108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:07 (8360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:10 (8360): No heartbeat from core client for 30 sec - exiting 12:51:11 (8360): No heartbeat from core client for 30 sec - exiting 12:51:12 (8360): No heartbeat from core client for 30 sec - exiting 12:51:13 (8360): No heartbeat from core client for 30 sec - exiting 13:01:08 (6248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:09:01 (7860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:34 (5244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:34:13 (6648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:34:14 (6648): No heartbeat from core client for 30 sec - exiting 13:44:13 (12728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:44:15 (12728): No heartbeat from core client for 30 sec - exiting 13:54:14 (3000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:54:15 (3000): No heartbeat from core client for 30 sec - exiting 13:54:16 (3000): No heartbeat from core client for 30 sec - exiting 13:55:13 (9200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:10:45 (11376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:10:49 (11376): No heartbeat from core client for 30 sec - exiting 14:10:50 (11376): No heartbeat from core client for 30 sec - exiting 14:10:51 (11376): No heartbeat from core client for 30 sec - exiting 14:10:52 (11376): No heartbeat from core client for 30 sec - exiting 14:10:54 (11376): No heartbeat from core client for 30 sec - exiting 14:10:55 (11376): No heartbeat from core client for 30 sec - exiting 14:20:51 (10620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:30:47 (1428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 15:09:01 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Jun 2013 09:24:51	330080	15857253	hadcm3n_n7fw_1920_40_008334551_1	25,920	42,025	1.6213