Task 15636382

Name	hadcm3n_3g7m_1980_40_008321245_0
Workunit	8472380
Created	24 Feb 2013, 18:38:14 UTC
Sent	24 Feb 2013, 19:42:14 UTC
Report deadline	27 May 2013, 3:09:25 UTC
Received	3 Mar 2013, 18:57:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1268402
Run time	3 days 10 hours 7 min 22 sec
CPU time	3 days 4 hours 26 min 9 sec
Validate state	Invalid
Credit	933.12
Device peak FLOPS	2.44 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:57:46 (7756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:50 (7756): No heartbeat from core client for 30 sec - exiting 08:57:51 (7756): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 09:16:45 (9056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:16:50 (9056): No heartbeat from core client for 30 sec - exiting 09:16:51 (9056): No heartbeat from core client for 30 sec - exiting 09:16:52 (9056): No heartbeat from core client for 30 sec - exiting 09:16:53 (9056): No heartbeat from core client for 30 sec - exiting 09:16:54 (9056): No heartbeat from core client for 30 sec - exiting 09:16:55 (9056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:49:57 (5408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:38 (5408): No heartbeat from core client for 30 sec - exiting 19:50:39 (5408): No heartbeat from core client for 30 sec - exiting 19:50:42 (5408): No heartbeat from core client for 30 sec - exiting 19:50:43 (5408): No heartbeat from core client for 30 sec - exiting 19:51:48 (5408):19:55:00 (7608): No heartbeat from core client for 30 sec - exiting 19:55:01 (7608): No heartbeat from core client for 30 sec - exiting 19:55:02 (7608): No heartbeat from core client for 30 sec - exiting 19:55:03 (7608): No heartbeat from core client for 30 sec - exiting 19:55:04 (7608): No heartbeat from core client for 30 sec - exiting 19:55:05 (7608): No heartbeat from core client for 30 sec - exiting 19:55:06 (7608): No heartbeat from core client for 30 sec - exiting 19:55:07 (7608): No heartbeat from core client for 30 sec - exiting 19:55:08 (7608): No heartbeat from core client for 30 sec - exiting 19:55:09 (7608): No heartbeat from core client for 30 sec - exiting 19:55:10 (7608): No heartbeat from core client for 30 sec - exiting 19:55:11 (7608): No heartbeat from core client for 30 sec - exiting 19:55:12 (7608): No heartbeat from core client for 30 sec - exiting 19:55:13 (7608): No heartbeat from core client for 30 sec - exiting 19:55:14 (7608): No heartbeat from core client for 30 sec - exiting 19:55:15 (7608): No heartbeat from core client for 30 sec - exiting 19:55:16 (7608): No heartbeat from core client for 30 sec - exiting 19:55:17 (7608): No heartbeat from core client for 30 sec - exiting 19:55:18 (7608): No heartbeat from core client for 30 sec - exiting 19:55:19 (7608): No heartbeat from core client for 30 sec - exiting 19:55:20 (7608): No heartbeat from core client for 30 sec - exiting 19:55:21 (7608): No heartbeat from core client for 30 sec - exiting 19:55:22 (7608): No heartbeat from core client for 30 sec - exiting 19:55:24 (7608): No heartbeat from core client for 30 sec - exiting 19:55:25 (7608): No heartbeat from core client for 30 sec - exiting 19:55:26 (7608): No heartbeat from core client for 30 sec - exiting 19:55:27 (7608): No heartbeat from core client for 30 sec - exiting 19:55:28 (7608): No heartbeat from core client for 30 sec - exiting 19:55:29 (7608): No heartbeat from core client for 30 sec - exiting 19:55:30 (7608): No heartbeat from core client for 30 sec - exiting 19:55:31 (7608): No heartbeat from core client for 30 sec - exiting 19:55:32 (7608): No heartbeat from core client for 30 sec - exiting 19:55:33 (7608): No heartbeat from core client for 30 sec - exiting 19:55:34 (7608): No heartbeat from core client for 30 sec - exiting 19:55:35 (7608): No heartbeat from core client for 30 sec - exiting 19:55:36 (7608): No heartbeat from core client for 30 sec - exiting 19:55:37 (7608): No heartbeat from core client for 30 sec - exiting 19:55:38 (7608): No heartbeat from core client for 30 sec - exiting 19:55:39 (7608): No heartbeat from core client for 30 sec - exiting 19:55:40 (7608): No heartbeat from core client for 30 sec - exiting 19:55:41 (7608): No heartbeat from core client for 30 sec - exiting 19:55:42 (7608): No heartbeat from core client for 30 sec - exiting 19:55:43 (7608): No heartbeat from core client for 30 sec - exiting 19:55:44 (7608): No heartbeat from core client for 30 sec - exiting 19:55:45 (7608): No heartbeat from core client for 30 sec - exiting 19:55:46 (7608): No heartbeat from core client for 30 sec - exiting 19:55:47 (7608): No heartbeat from core client for 30 sec - exiting 19:55:48 (7608): No heartbeat from core client for 30 sec - exiting 19:55:49 (7608): No heartbeat from core client for 30 sec - exiting 19:55:50 (7608): No heartbeat from core client for 30 sec - exiting 19:55:51 (7608): No heartbeat from core client for 30 sec - exiting 19:55:52 (7608): No heartbeat from core client for 30 sec - exiting 19:55:53 (7608): No heartbeat from core client for 30 sec - exiting 19:55:54 (7608): No heartbeat from core client for 30 sec - exiting 19:55:55 (7608): No heartbeat from core client for 30 sec - exiting 19:55:56 (7608): No heartbeat from core client for 30 sec - exiting 19:55:57 (7608): No heartbeat from core client for 30 sec - exiting 19:55:58 (7608): No heartbeat from core client for 30 sec - exiting 19:55:59 (7608): No heartbeat from core client for 30 sec - exiting 19:56:00 (7608): No heartbeat from core client for 30 sec - exiting 19:56:01 (7608): No heartbeat from core client for 30 sec - exiting 19:56:02 (7608): No heartbeat from core client for 30 sec - exiting 19:56:03 (7608): No heartbeat from core client for 30 sec - exiting 19:56:04 (7608): No heartbeat from core client for 30 sec - exiting 19:56:05 (7608): No heartbeat from core client for 30 sec - exiting 19:56:06 (7608): No heartbeat from core client for 30 sec - exiting 19:56:07 (7608): No heartbeat from core client for 30 sec - exiting 19:56:08 (7608): No heartbeat from core client for 30 sec - exiting 19:56:09 (7608): No heartbeat from core client for 30 sec - exiting 19:56:10 (7608): No heartbeat from core client for 30 sec - exiting 19:56:11 (7608): No heartbeat from core client for 30 sec - exiting 19:56:12 (7608): No heartbeat from core client for 30 sec - exiting 19:56:13 (7608): No heartbeat from core client for 30 sec - exiting 19:56:14 (7608): No heartbeat from core client for 30 sec - exiting 19:56:15 (7608): No heartbeat from core client for 30 sec - exiting 19:56:16 (7608): No heartbeat from core client for 30 sec - exiting 19:56:17 (7608): No heartbeat from core client for 30 sec - exiting 19:56:18 (7608): No heartbeat from core client for 30 sec - exiting 19:56:19 (7608): No heartbeat from core client for 30 sec - exiting 19:56:20 (7608): No heartbeat from core client for 30 sec - exiting 19:56:21 (7608): No heartbeat from core client for 30 sec - exiting 19:56:22 (7608): No heartbeat from core client for 30 sec - exiting 19:56:23 (7608): No heartbeat from core client for 30 sec - exiting 19:56:24 (7608): No heartbeat from core client for 30 sec - exiting 19:56:25 (7608): No heartbeat from core client for 30 sec - exiting 19:56:26 (7608): No heartbeat from core client for 30 sec - exiting 19:56:27 (7608): No heartbeat from core client for 30 sec - exiting 19:56:28 (7608): No heartbeat from core client for 30 sec - exiting 19:56:29 (7608): No heartbeat from core client for 30 sec - exiting 19:56:30 (7608): No heartbeat from core client for 30 sec - exiting 19:56:31 (7608): No heartbeat from core client for 30 sec - exiting 19:56:32 (7608): No heartbeat from core client for 30 sec - exiting 19:56:33 (7608): No heartbeat from core client for 30 sec - exiting 19:56:34 (7608): No heartbeat from core client for 30 sec - exiting 19:56:35 (7608): No heartbeat from core client for 30 sec - exiting 19:56:36 (7608): No heartbeat from core client for 30 sec - exiting 19:56:37 (7608): No heartbeat from core client for 30 sec - exiting 19:56:38 (7608): No heartbeat from core client for 30 sec - exiting 19:56:39 (7608): No heartbeat from core client for 30 sec - exiting 19:56:40 (7608): No heartbeat from core client for 30 sec - exiting 19:56:41 (7608): No heartbeat from core client for 30 sec - exiting 19:56:42 (7608): No heartbeat from core client for 30 sec - exiting 19:56:43 (7608): No heartbeat from core client for 30 sec - exiting 19:56:44 (7608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:15 (7696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:17 (7696): No heartbeat from core client for 30 sec - exiting 20:00:18 (7696): No heartbeat from core client for 30 sec - exiting 20:00:19 (7696): No heartbeat from core client for 30 sec - exiting 20:11:05 (8324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:07 (8324): No heartbeat from core client for 30 sec - exiting 20:11:08 (8324): No heartbeat from core client for 30 sec - exiting 21:19:33 (7300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:11:46 (6568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:12:00 (6568): No heartbeat from core client for 30 sec - exiting 01:12:01 (6568): No heartbeat from core client for 30 sec - exiting 01:12:02 (6568): No heartbeat from core client for 30 sec - exiting 01:12:04 (6568): No heartbeat from core client for 30 sec - exiting 01:12:05 (6568): No heartbeat from core client for 30 sec - exiting 01:12:06 (6568): No heartbeat from core client for 30 sec - exiting 01:12:09 (6568): No heartbeat from core client for 30 sec - exiting 01:12:10 (6568): No heartbeat from core client for 30 sec - exiting 01:12:11 (6568): No heartbeat from core client for 30 sec - exiting 01:12:12 (6568): No heartbeat from core client for 30 sec - exiting 01:12:13 (6568): No heartbeat from core client for 30 sec - exiting 01:12:14 (6568): No heartbeat from core client for 30 sec - exiting 01:12:15 (6568): No heartbeat from core client for 30 sec - exiting 01:12:16 (6568): No heartbeat from core client for 30 sec - exiting 01:12:17 (6568): No heartbeat from core client for 30 sec - exiting 01:12:18 (6568): No heartbeat from core client for 30 sec - exiting 01:12:19 (6568): No heartbeat from core client for 30 sec - exiting 01:12:20 (6568): No heartbeat from core client for 30 sec - exiting 01:12:22 (6568): No heartbeat from core client for 30 sec - exiting 02:22:13 (7044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:23:40 (7044): No heartbeat from core client for 30 sec - exiting 02:23:41 (7044): No heartbeat from core client for 30 sec - exiting 02:23:42 (7044): No heartbeat from core client for 30 sec - exiting 02:23:43 (7044): No heartbeat from core client for 30 sec - exiting 02:23:44 (7044): No heartbeat from core client for 30 sec - exiting 02:23:45 (7044): No heartbeat from core client for 30 sec - exiting 02:23:46 (7044): No heartbeat from core client for 30 sec - exiting 02:23:47 (7044): No heartbeat from core client for 30 sec - exiting 02:23:48 (7044): No heartbeat from core client for 30 sec - exiting 02:23:51 (7044): No heartbeat from core client for 30 sec - exiting 02:24:28 (7044): No heartbeat from core client for 30 sec - exiting 02:24:29 (7044): No heartbeat from core client for 30 sec - exiting 02:24:30 (7044): No heartbeat from core client for 30 sec - exiting 02:24:31 (7044): No heartbeat from core client for 30 sec - exiting 02:24:33 (7044): No heartbeat from core client for 30 sec - exiting 02:24:34 (7044): No heartbeat from core client for 30 sec - exiting 02:30:55 (7460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:30:59 (7460): No heartbeat from core client for 30 sec - exiting 02:31:00 (7460): No heartbeat from core client for 30 sec - exiting 02:31:01 (7460): No heartbeat from core client for 30 sec - exiting 02:31:02 (7460): No heartbeat from core client for 30 sec - exiting 02:31:03 (7460): No heartbeat from core client for 30 sec - exiting 02:31:04 (7460): No heartbeat from core client for 30 sec - exiting 02:31:05 (7460): No heartbeat from core client for 30 sec - exiting 02:31:06 (7460): No heartbeat from core client for 30 sec - exiting 02:37:59 (7664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:38:09 (7664): No heartbeat from core client for 30 sec - exiting 02:38:11 (7664): No heartbeat from core client for 30 sec - exiting 02:38:13 (7664): No heartbeat from core client for 30 sec - exiting 02:38:14 (7664): No heartbeat from core client for 30 sec - exiting 02:38:15 (7664): No heartbeat from core client for 30 sec - exiting 02:38:16 (7664): No heartbeat from core client for 30 sec - exiting 02:45:37 (7700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:46:26 (7700): No heartbeat from core client for 30 sec - exiting 02:46:30 (7700): No heartbeat from core client for 30 sec - exiting 02:46:32 (7700): No heartbeat from core client for 30 sec - exiting 02:46:33 (7700): No heartbeat from core client for 30 sec - exiting 02:46:35 (7700): No heartbeat from core client for 30 sec - exiting 02:46:36 (7700): No heartbeat from core client for 30 sec - exiting 02:46:38 (7700): No heartbeat from core client for 30 sec - exiting 02:46:39 (7700): No heartbeat from core client for 30 sec - exiting 02:46:40 (7700): No heartbeat from core client for 30 sec - exiting 02:46:41 (7700): No heartbeat from core client for 30 sec - exiting 02:46:42 (7700): No heartbeat from core client for 30 sec - exiting 02:46:43 (7700): No heartbeat from core client for 30 sec - exiting 02:46:44 (7700): No heartbeat from core client for 30 sec - exiting 02:46:45 (7700): No heartbeat from core client for 30 sec - exiting 02:46:47 (7700): No heartbeat from core client for 30 sec - exiting 02:46:48 (7700): No heartbeat from core client for 30 sec - exiting 02:46:50 (7700): No heartbeat from core client for 30 sec - exiting 02:46:51 (7700): No heartbeat from core client for 30 sec - exiting 02:46:52 (7700): No heartbeat from core client for 30 sec - exiting 02:46:53 (7700): No heartbeat from core client for 30 sec - exiting 02:46:54 (7700): No heartbeat from core client for 30 sec - exiting 02:46:55 (7700): No heartbeat from core client for 30 sec - exiting 02:46:57 (7700): No heartbeat from core client for 30 sec - exiting 02:47:01 (7700): No heartbeat from core client for 30 sec - exiting 02:47:02 (7700): No heartbeat from core client for 30 sec - exiting 02:47:04 (7700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:36:41 (6888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:36:44 (6888): No heartbeat from core client for 30 sec - exiting 01:36:45 (6888): No heartbeat from core client for 30 sec - exiting 01:36:46 (6888): No heartbeat from core client for 30 sec - exiting 01:36:47 (6888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:58:08 (1096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:11 (1096): No heartbeat from core client for 30 sec - exiting 18:58:12 (1096): No heartbeat from core client for 30 sec - exiting 18:58:13 (1096): No heartbeat from core client for 30 sec - exiting 18:58:14 (1096): No heartbeat from core client for 30 sec - exiting 18:58:15 (1096): No heartbeat from core client for 30 sec - exiting 18:58:16 (1096): No heartbeat from core client for 30 sec - exiting 18:58:17 (1096): No heartbeat from core client for 30 sec - exiting 18:58:18 (1096): No heartbeat from core client for 30 sec - exiting 18:58:19 (1096): No heartbeat from core client for 30 sec - exiting 18:58:20 (1096): No heartbeat from core client for 30 sec - exiting 18:58:21 (1096): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:23:23 (6856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:29 (6856): No heartbeat from core client for 30 sec - exiting 01:23:31 (6856): No heartbeat from core client for 30 sec - exiting 01:23:32 (6856): No heartbeat from core client for 30 sec - exiting 01:23:33 (6856): No heartbeat from core client for 30 sec - exiting 01:23:34 (6856): No heartbeat from core client for 30 sec - exiting 01:23:35 (6856): No heartbeat from core client for 30 sec - exiting 01:23:36 (6856): No heartbeat from core client for 30 sec - exiting 01:23:37 (6856): No heartbeat from core client for 30 sec - exiting 01:23:38 (6856): No heartbeat from core client for 30 sec - exiting 01:23:39 (6856): No heartbeat from core client for 30 sec - exiting 01:23:41 (6856): No heartbeat from core client for 30 sec - exiting 01:23:42 (6856): No heartbeat from core client for 30 sec - exiting 01:23:43 (6856): No heartbeat from core client for 30 sec - exiting 01:23:44 (6856): No heartbeat from core client for 30 sec - exiting 01:23:45 (6856): No heartbeat from core client for 30 sec - exiting 01:23:46 (6856): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 01:26:42 (6140): No heartbeat from core client for 30 sec - exiting 01:26:43 (6140): No heartbeat from core client for 30 sec - exiting 01:26:44 (6140): No heartbeat from core client for 30 sec - exiting 01:26:45 (6140): No heartbeat from core client for 30 sec - exiting 01:26:46 (6140): No heartbeat from core client for 30 sec - exiting 01:26:47 (6140): No heartbeat from core client for 30 sec - exiting 01:26:48 (6140): No heartbeat from core client for 30 sec - exiting 01:26:49 (6140): No heartbeat from core client for 30 sec - exiting 01:26:50 (6140): No heartbeat from core client for 30 sec - exiting 01:26:51 (6140): No heartbeat from core client for 30 sec - exiting 01:26:52 (6140): No heartbeat from core client for 30 sec - exiting 01:26:53 (6140): No heartbeat from core client for 30 sec - exiting 01:26:54 (6140): No heartbeat from core client for 30 sec - exiting 01:26:55 (6140): No heartbeat from core client for 30 sec - exiting 01:26:56 (6140): No heartbeat from core client for 30 sec - exiting 01:26:57 (6140): No heartbeat from core client for 30 sec - exiting 01:26:58 (6140): No heartbeat from core client for 30 sec - exiting 01:26:59 (6140): No heartbeat from core client for 30 sec - exiting 01:27:00 (6140): No heartbeat from core client for 30 sec - exiting 01:27:01 (6140): No heartbeat from core client for 30 sec - exiting 01:27:02 (6140): No heartbeat from core client for 30 sec - exiting 01:27:03 (6140): No heartbeat from core client for 30 sec - exiting 01:27:04 (6140): No heartbeat from core client for 30 sec - exiting 01:27:05 (6140): No heartbeat from core client for 30 sec - exiting 01:27:06 (6140): No heartbeat from core client for 30 sec - exiting 01:27:07 (6140): No heartbeat from core client for 30 sec - exiting 01:27:08 (6140): No heartbeat from core client for 30 sec - exiting 01:27:09 (6140): No heartbeat from core client for 30 sec - exiting 01:27:10 (6140): No heartbeat from core client for 30 sec - exiting 01:27:11 (6140): No heartbeat from core client for 30 sec - exiting 01:27:12 (6140): No heartbeat from core client for 30 sec - exiting 01:27:13 (6140): No heartbeat from core client for 30 sec - exiting 01:27:14 (6140): No heartbeat from core client for 30 sec - exiting 01:27:15 (6140): No heartbeat from core client for 30 sec - exiting 01:27:16 (6140): No heartbeat from core client for 30 sec - exiting 01:27:17 (6140): No heartbeat from core client for 30 sec - exiting 01:27:18 (6140): No heartbeat from core client for 30 sec - exiting 01:27:19 (6140): No heartbeat from core client for 30 sec - exiting 01:27:20 (6140): No heartbeat from core client for 30 sec - exiting 01:27:21 (6140): No heartbeat from core client for 30 sec - exiting 01:27:22 (6140): No heartbeat from core client for 30 sec - exiting 01:27:23 (6140): No heartbeat from core client for 30 sec - exiting 01:27:24 (6140): No heartbeat from core client for 30 sec - exiting 01:27:25 (6140): No heartbeat from core client for 30 sec - exiting 01:27:26 (6140): No heartbeat from core client for 30 sec - exiting 01:27:27 (6140): No heartbeat from core client for 30 sec - exiting 01:27:28 (6140): No heartbeat from core client for 30 sec - exiting 01:27:29 (6140): No heartbeat from core client for 30 sec - exiting 01:27:30 (6140): No heartbeat from core client for 30 sec - exiting 01:27:31 (6140): No heartbeat from core client for 30 sec - exiting 01:27:32 (6140): No heartbeat from core client for 30 sec - exiting 01:27:33 (6140): No heartbeat from core client for 30 sec - exiting 01:27:34 (6140): No heartbeat from core client for 30 sec - exiting 01:27:35 (6140): No heartbeat from core client for 30 sec - exiting 01:27:36 (6140): No heartbeat from core client for 30 sec - exiting 01:27:37 (6140): No heartbeat from core client for 30 sec - exiting 01:27:38 (6140): No heartbeat from core client for 30 sec - exiting 01:27:39 (6140): No heartbeat from core client for 30 sec - exiting 01:27:40 (6140): No heartbeat from core client for 30 sec - exiting 01:27:41 (6140): No heartbeat from core client for 30 sec - exiting 01:27:42 (6140): No heartbeat from core client for 30 sec - exiting 01:27:43 (6140): No heartbeat from core client for 30 sec - exiting 01:27:44 (6140): No heartbeat from core client for 30 sec - exiting 01:27:45 (6140): No heartbeat from core client for 30 sec - exiting 01:27:46 (6140): No heartbeat from core client for 30 sec - exiting 01:27:47 (6140): No heartbeat from core client for 30 sec - exiting 01:27:48 (6140): No heartbeat from core client for 30 sec - exiting 01:27:49 (6140): No heartbeat from core client for 30 sec - exiting 01:27:50 (6140): No heartbeat from core client for 30 sec - exiting 01:27:51 (6140): No heartbeat from core client for 30 sec - exiting 01:27:52 (6140): No heartbeat from core client for 30 sec - exiting 01:27:53 (6140): No heartbeat from core client for 30 sec - exiting 01:27:54 (6140): No heartbeat from core client for 30 sec - exiting 01:27:55 (6140): No heartbeat from core client for 30 sec - exiting 01:27:56 (6140): No heartbeat from core client for 30 sec - exiting 01:27:57 (6140): No heartbeat from core client for 30 sec - exiting 01:27:58 (6140): No heartbeat from core client for 30 sec - exiting 01:27:59 (6140): No heartbeat from core client for 30 sec - exiting 01:28:00 (6140): No heartbeat from core client for 30 sec - exiting 01:28:01 (6140): No heartbeat from core client for 30 sec - exiting 01:28:02 (6140): No heartbeat from core client for 30 sec - exiting 01:28:03 (6140): No heartbeat from core client for 30 sec - exiting 01:28:05 (6140): No heartbeat from core client for 30 sec - exiting 01:28:06 (6140): No heartbeat from core client for 30 sec - exiting 01:28:07 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:34:54 (9904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:34:59 (9904): No heartbeat from core client for 30 sec - exiting 02:35:00 (9904): No heartbeat from core client for 30 sec - exiting 02:35:01 (9904): No heartbeat from core client for 30 sec - exiting 02:35:02 (9904): No heartbeat from core client for 30 sec - exiting 02:35:03 (9904): No heartbeat from core client for 30 sec - exiting 02:35:04 (9904): No heartbeat from core client for 30 sec - exiting 02:35:05 (9904): No heartbeat from core client for 30 sec - exiting 02:35:06 (9904): No heartbeat from core client for 30 sec - exiting 02:35:07 (9904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 03:52:13 (9772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:52:17 (9772): No heartbeat from core client for 30 sec - exiting 03:52:18 (9772): No heartbeat from core client for 30 sec - exiting 03:52:19 (9772): No heartbeat from core client for 30 sec - exiting 03:52:20 (9772): No heartbeat from core client for 30 sec - exiting 03:52:21 (9772): No heartbeat from core client for 30 sec - exiting 03:52:22 (9772): No heartbeat from core client for 30 sec - exiting 03:52:23 (9772): No heartbeat from core client for 30 sec - exiting 03:52:24 (9772): No heartbeat from core client for 30 sec - exiting 03:52:26 (9772): No heartbeat from core client for 30 sec - exiting 03:52:27 (9772): No heartbeat from core client for 30 sec - exiting 03:52:28 (9772): No heartbeat from core client for 30 sec - exiting 03:52:29 (9772): No heartbeat from core client for 30 sec - exiting 03:52:30 (9772): No heartbeat from core client for 30 sec - exiting 04:00:34 (5012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4832, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4832, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9800, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12284, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:34:15 (7276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:34:23 (7276): No heartbeat from core client for 30 sec - exiting 14:34:24 (7276): No heartbeat from core client for 30 sec - exiting 14:34:25 (7276): No heartbeat from core client for 30 sec - exiting 14:34:26 (7276): No heartbeat from core client for 30 sec - exiting 14:34:28 (7276): No heartbeat from core client for 30 sec - exiting 14:34:30 (7276): No heartbeat from core client for 30 sec - exiting 14:34:31 (7276): No heartbeat from core client for 30 sec - exiting 14:34:32 (7276): No heartbeat from core client for 30 sec - exiting 14:34:33 (7276): No heartbeat from core client for 30 sec - exiting 14:34:34 (7276): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5928, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5928, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Mar 2013 00:54:51	1268402	15636382	hadcm3n_3g7m_1980_40_008321245_0	77,760	213,219	2.7420
28 Feb 2013 18:59:27	1268402	15636382	hadcm3n_3g7m_1980_40_008321245_0	51,840	142,953	2.7576
26 Feb 2013 17:51:45	1268402	15636382	hadcm3n_3g7m_1980_40_008321245_0	25,920	70,972	2.7381