Task 15650799

Name	hadcm3n_o3d8_2140_40_008282229_2
Workunit	8433364
Created	6 Mar 2013, 8:21:31 UTC
Sent	6 Mar 2013, 8:21:52 UTC
Report deadline	5 Jun 2013, 15:49:03 UTC
Received	10 May 2013, 17:02:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1046143
Run time	1 days 10 hours 20 min 8 sec
CPU time	1 days 10 hours 20 min 8 sec
Validate state	Invalid
Credit	933.12
Device peak FLOPS	3.02 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.10.45</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:02:35 (2012): No heartbeat from core client for 30 sec - exiting 08:02:36 (2012): No heartbeat from core client for 30 sec - exiting 08:02:37 (2012): No heartbeat from core client for 30 sec - exiting 08:02:38 (2012): No heartbeat from core client for 30 sec - exiting 08:02:39 (2012): No heartbeat from core client for 30 sec - exiting 08:02:40 (2012): No heartbeat from core client for 30 sec - exiting 08:02:41 (2012): No heartbeat from core client for 30 sec - exiting 08:02:42 (2012): No heartbeat from core client for 30 sec - exiting 08:02:43 (2012): No heartbeat from core client for 30 sec - exiting 08:02:44 (2012): No heartbeat from core client for 30 sec - exiting 08:02:45 (2012): No heartbeat from core client for 30 sec - exiting 08:02:46 (2012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:22:03 (4796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:27 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:33:27 (3748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:59:31 (3996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:24 (3512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:49 (3512): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4120, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1064, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 12:18:06 (2104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:18:40 (5944): No heartbeat from core client for 30 sec - exiting 12:18:41 (5944): No heartbeat from core client for 30 sec - exiting 12:18:42 (5944): No heartbeat from core client for 30 sec - exiting 12:18:45 (5944): No heartbeat from core client for 30 sec - exiting 12:18:49 (5944): No heartbeat from core client for 30 sec - exiting 12:18:50 (5944): No heartbeat from core client for 30 sec - exiting 12:18:54 (5944): No heartbeat from core client for 30 sec - exiting 12:18:55 (5944): No heartbeat from core client for 30 sec - exiting 12:18:56 (5944): No heartbeat from core client for 30 sec - exiting 12:18:57 (5944): No heartbeat from core client for 30 sec - exiting 12:18:58 (5944): No heartbeat from core client for 30 sec - exiting 12:19:00 (5944): No heartbeat from core client for 30 sec - exiting 12:19:01 (5944): No heartbeat from core client for 30 sec - exiting 12:19:10 (5944): No heartbeat from core client for 30 sec - exiting 12:19:12 (5944): No heartbeat from core client for 30 sec - exiting 12:19:13 (5944): No heartbeat from core client for 30 sec - exiting 12:19:17 (5944): No heartbeat from core client for 30 sec - exiting 12:19:18 (5944): No heartbeat from core client for 30 sec - exiting 12:19:19 (5944): No heartbeat from core client for 30 sec - exiting 12:19:20 (5944): No heartbeat from core client for 30 sec - exiting 12:19:21 (5944): No heartbeat from core client for 30 sec - exiting 12:19:22 (5944): No heartbeat from core client for 30 sec - exiting 12:19:23 (5944): No heartbeat from core client for 30 sec - exiting 12:19:24 (5944): No heartbeat from core client for 30 sec - exiting 12:19:25 (5944): No heartbeat from core client for 30 sec - exiting 12:19:34 (5944): No heartbeat from core client for 30 sec - exiting 12:19:40 (5944): No heartbeat from core client for 30 sec - exiting 12:19:41 (5944): No heartbeat from core client for 30 sec - exiting 12:19:47 (5944): No heartbeat from core client for 30 sec - exiting 12:19:48 (5944): No heartbeat from core client for 30 sec - exiting 12:19:56 (5944): No heartbeat from core client for 30 sec - exiting 12:19:57 (5944): No heartbeat from core client for 30 sec - exiting 12:19:58 (5944): No heartbeat from core client for 30 sec - exiting 12:19:59 (5944): No heartbeat from core client for 30 sec - exiting 12:20:00 (5944): No heartbeat from core client for 30 sec - exiting 12:20:02 (5944): No heartbeat from core client for 30 sec - exiting 12:20:03 (5944): No heartbeat from core client for 30 sec - exiting 12:20:04 (5944): No heartbeat from core client for 30 sec - exiting 12:20:05 (5944): No heartbeat from core client for 30 sec - exiting 12:20:06 (5944):CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting 12:32:52 (2116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:31 (2828): No heartbeat from core client for 30 sec - exiting 12:34:33 (2828): No heartbeat from core client for 30 sec - exiting 12:34:34 (2828): No heartbeat from core client for 30 sec - exiting 12:34:35 (2828): No heartbeat from core client for 30 sec - exiting 12:34:36 (2828): No heartbeat from core client for 30 sec - exiting 12:34:37 (2828): No heartbeat from core client for 30 sec - exiting 12:34:39 (2828): No heartbeat from core client for 30 sec - exiting 12:34:40 (2828): No heartbeat from core client for 30 sec - exiting 12:34:41 (2828): No heartbeat from core client for 30 sec - exiting 12:34:42 (2828): No heartbeat from core client for 30 sec - exiting 12:34:43 (2828): No heartbeat from core client for 30 sec - exiting 12:34:45 (2828): No heartbeat from core client for 30 sec - exiting 12:34:46 (2828): No heartbeat from core client for 30 sec - exiting 12:34:47 (2828): No heartbeat from core client for 30 sec - exiting 12:34:49 (2828): No heartbeat from core client for 30 sec - exiting 12:34:50 (2828): No heartbeat from core client for 30 sec - exiting 12:34:51 (2828): No heartbeat from core client for 30 sec - exiting 12:34:52 (2828): No heartbeat from core client for 30 sec - exiting 12:34:53 (2828): No heartbeat from core client for 30 sec - exiting 12:34:55 (2828): No heartbeat from core client for 30 sec - exiting 12:34:57 (2828): No heartbeat from core client for 30 sec - exiting 12:34:58 (2828): No heartbeat from core client for 30 sec - exiting 12:34:59 (2828): No heartbeat from core client for 30 sec - exiting 12:35:00 (2828): No heartbeat from core client for 30 sec - exiting 12:35:01 (2828): No heartbeat from core client for 30 sec - exiting 12:35:03 (2828): No heartbeat from core client for 30 sec - exiting 12:35:04 (2828): No heartbeat from core client for 30 sec - exiting 12:35:05 (2828): No heartbeat from core client for 30 sec - exiting 12:35:07 (2828): No heartbeat from core client for 30 sec - exiting 12:35:08 (2828): No heartbeat from core client for 30 sec - exiting 12:35:09 (2828): No heartbeat from core client for 30 sec - exiting 12:35:10 (2828): No heartbeat from core client for 30 sec - exiting 12:35:11 (2828): No heartbeat from core client for 30 sec - exiting 12:35:12 (2828): No heartbeat from core client for 30 sec - exiting 12:35:13 (2828): No heartbeat from core client for 30 sec - exiting 12:35:14 (2828): No heartbeat from core client for 30 sec - exiting 12:35:15 (2828): No heartbeat from core client for 30 sec - exiting 12:35:16 (2828): No heartbeat from core client for 30 sec - exiting 12:35:17 (2828): No heartbeat from core client for 30 sec - exiting 12:35:19 (2828): No heartbeat from core client for 30 sec - exiting 12:35:20 (2828): No heartbeat from core client for 30 sec - exiting 12:35:21 (2828): No heartbeat from core client for 30 sec - exiting 12:35:22 (2828): No heartbeat from core client for 30 sec - exiting 12:35:24 (2828): No heartbeat from core client for 30 sec - exiting 12:35:25 (2828): No heartbeat from core client for 30 sec - exiting 12:35:26 (2828): No heartbeat from core client for 30 sec - exiting 12:35:28 (2828): No heartbeat from core client for 30 sec - exiting 12:35:29 (2828): No heartbeat from core client for 30 sec - exiting 12:35:30 (2828): No heartbeat from core client for 30 sec - exiting 12:35:31 (2828): No heartbeat from core client for 30 sec - exiting 12:35:32 (2828): No heartbeat from core client for 30 sec - exiting 12:35:34 (2828): No heartbeat from core client for 30 sec - exiting 12:35:35 (2828): No heartbeat from core client for 30 sec - exiting 12:35:36 (2828): No heartbeat from core client for 30 sec - exiting 12:35:38 (2828): No heartbeat from core client for 30 sec - exiting 12:35:39 (2828): No heartbeat from core client for 30 sec - exiting 12:35:40 (2828): No heartbeat from core client for 30 sec - exiting 12:35:41 (2828): No heartbeat from core client for 30 sec - exiting 12:35:42 (2828): No heartbeat from core client for 30 sec - exiting 12:35:43 (2828): No heartbeat from core client for 30 sec - exiting 12:35:44 (2828): No heartbeat from core client for 30 sec - exiting 12:35:45 (2828): No heartbeat from core client for 30 sec - exiting 12:35:46 (2828): No heartbeat from core client for 30 sec - exiting 12:35:48 (2828): No heartbeat from core client for 30 sec - exiting 12:35:49 (2828): No heartbeat from core client for 30 sec - exiting 12:35:51 (2828): No heartbeat from core client for 30 sec - exiting 12:35:52 (2828): No heartbeat from core client for 30 sec - exiting 12:35:53 (2828): No heartbeat from core client for 30 sec - exiting 12:35:54 (2828): No heartbeat from core client for 30 sec - exiting 12:35:55 (2828): No heartbeat from core client for 30 sec - exiting 12:35:56 (2828): No heartbeat from core client for 30 sec - exiting 12:35:58 (2828): No heartbeat from core client for 30 sec - exiting 12:35:59 (2828): No heartbeat from core client for 30 sec - exiting 12:36:00 (2828): No heartbeat from core client for 30 sec - exiting 12:36:01 (2828): No heartbeat from core client for 30 sec - exiting 12:36:02 (2828): No heartbeat from core client for 30 sec - exiting 12:36:04 (2828): No heartbeat from core client for 30 sec - exiting 12:36:05 (2828): No heartbeat from core client for 30 sec - exiting 12:36:06 (2828): No heartbeat from core client for 30 sec - exiting 12:36:07 (2828): No heartbeat from core client for 30 sec - exiting 12:36:08 (2828): No heartbeat from core client for 30 sec - exiting 12:36:09 (2828): No heartbeat from core client for 30 sec - exiting 12:36:10 (2828): No heartbeat from core client for 30 sec - exiting 12:36:12 (2828): No heartbeat from core client for 30 sec - exiting 12:36:13 (2828): No heartbeat from core client for 30 sec - exiting 12:36:14 (2828): No heartbeat from core client for 30 sec - exiting 12:36:15 (2828): No heartbeat from core client for 30 sec - exiting 12:36:16 (2828): No heartbeat from core client for 30 sec - exiting 12:36:17 (2828): No heartbeat from core client for 30 sec - exiting 12:36:18 (2828): No heartbeat from core client for 30 sec - exiting 12:36:19 (2828): No heartbeat from core client for 30 sec - exiting 12:36:20 (2828): No heartbeat from core client for 30 sec - exiting 12:36:21 (2828): No heartbeat from core client for 30 sec - exiting 12:36:22 (2828): No heartbeat from core client for 30 sec - exiting 12:36:23 (2828): No heartbeat from core client for 30 sec - exiting 12:36:24 (2828): No heartbeat from core client for 30 sec - exiting 12:36:25 (2828): No heartbeat from core client for 30 sec - exiting 12:36:26 (2828): No heartbeat from core client for 30 sec - exiting 12:36:27 (2828): No heartbeat from core client for 30 sec - exiting 12:36:28 (2828): No heartbeat from core client for 30 sec - exiting 12:36:29 (2828): No heartbeat from core client for 30 sec - exiting 12:36:30 (2828): No heartbeat from core client for 30 sec - exiting 12:36:31 (2828): No heartbeat from core client for 30 sec - exiting 12:36:32 (2828): No heartbeat from core client for 30 sec - exiting 12:36:33 (2828): No heartbeat from core client for 30 sec - exiting 12:36:35 (2828): No heartbeat from core client for 30 sec - exiting 12:36:36 (2828): No heartbeat from core client for 30 sec - exiting 12:36:37 (2828): No heartbeat from core client for 30 sec - exiting 12:36:38 (2828): No heartbeat from core client for 30 sec - exiting 12:36:39 (2828): No heartbeat from core client for 30 sec - exiting 12:36:40 (2828): No heartbeat from core client for 30 sec - exiting 12:36:41 (2828): No heartbeat from core client for 30 sec - exiting 12:36:42 (2828): No heartbeat from core client for 30 sec - exiting 12:36:43 (2828): No heartbeat from core client for 30 sec - exiting 12:36:45 (2828): No heartbeat from core client for 30 sec - exiting 12:36:46 (2828): No heartbeat from core client for 30 sec - exiting 12:36:47 (2828): No heartbeat from core client for 30 sec - exiting 12:36:48 (2828): No heartbeat from core client for 30 sec - exiting 12:36:49 (2828): No heartbeat from core client for 30 sec - exiting 12:36:50 (2828): No heartbeat from core client for 30 sec - exiting 12:36:51 (2828): No heartbeat from core client for 30 sec - exiting 12:36:53 (2828): No heartbeat from core client for 30 sec - exiting 12:36:54 (2828): No heartbeat from core client for 30 sec - exiting 12:36:55 (2828): No heartbeat from core client for 30 sec - exiting 12:36:56 (2828): No heartbeat from core client for 30 sec - exiting 12:36:57 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:26 (4928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:48:34 (2760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:49:08 (5936): No heartbeat from core client for 30 sec - exiting 00:49:09 (5936): No heartbeat from core client for 30 sec - exiting 00:49:10 (5936): No heartbeat from core client for 30 sec - exiting 00:49:11 (5936): No heartbeat from core client for 30 sec - exiting 00:49:12 (5936): No heartbeat from core client for 30 sec - exiting 00:49:14 (5936): No heartbeat from core client for 30 sec - exiting 00:49:15 (5936): No heartbeat from core client for 30 sec - exiting 00:49:42 (5936): No heartbeat from core client for 30 sec - exiting 00:49:43 (5936): No heartbeat from core client for 30 sec - exiting 00:49:45 (5936): No heartbeat from core client for 30 sec - exiting 00:49:46 (5936): No heartbeat from core client for 30 sec - exiting 00:49:47 (5936): No heartbeat from core client for 30 sec - exiting 00:49:48 (5936): No heartbeat from core client for 30 sec - exiting 00:49:49 (5936): No heartbeat from core client for 30 sec - exiting 00:50:10 (5936): No heartbeat from core client for 30 sec - exiting 00:50:34 (5936): No heartbeat from core client for 30 sec - exiting 00:50:36 (5936): No heartbeat from core client for 30 sec - exiting 00:51:10 (5936): No heartbeat from core client for 30 sec - exiting 00:51:11 (5936): No heartbeat from core client for 30 sec - exiting 00:51:12 (5936): No heartbeat from core client for 30 sec - exiting 00:51:13 (5936): No heartbeat from core client for 30 sec - exiting 00:51:23 (5936): No heartbeat from core client for 30 sec - exiting 00:51:24 (5936): No heartbeat from core client for 30 sec - exiting 00:51:25 (5936): No heartbeat from core client for 30 sec - exiting 00:51:42 (5936): No heartbeat from core client for 30 sec - exiting 00:52:08 (5936): No heartbeat from core client for 30 sec - exiting 00:52:09 (5936): No heartbeat from core client for 30 sec - exiting 00:52:10 (5936): No heartbeat from core client for 30 sec - exiting 00:52:11 (5936): No heartbeat from core client for 30 sec - exiting 00:52:12 (5936): No heartbeat from core client for 30 sec - exiting 00:52:13 (5936): No heartbeat from core client for 30 sec - exiting 00:52:15 (5936): No heartbeat from core client for 30 sec - exiting 00:52:16 (5936): No heartbeat from core client for 30 sec - exiting 00:52:17 (5936): No heartbeat from core client for 30 sec - exiting 00:52:45 (5936): No heartbeat from core client for 30 sec - exiting 00:52:46 (5936): No heartbeat from core client for 30 sec - exiting 00:52:47 (5936): No heartbeat from core client for 30 sec - exiting 00:52:49 (5936): No heartbeat from core client for 30 sec - exiting 00:53:22 (5936): No heartbeat from core client for 30 sec - exiting 00:53:27 (5936): No heartbeat from core client for 30 sec - exiting 00:53:36 (5936): No heartbeat from core client for 30 sec - exiting 00:53:37 (5936): No heartbeat from core client for 30 sec - exiting 00:53:39 (5936): No heartbeat from core client for 30 sec - exiting 00:53:40 (5936): No heartbeat from core client for 30 sec - exiting 00:54:15 (5936): No heartbeat from core client for 30 sec - exiting 00:54:28 (5936): No heartbeat from core client for 30 sec - exiting 00:54:40 (5936): No heartbeat from core client for 30 sec - exiting 00:54:41 (5936): No heartbeat from core client for 30 sec - exiting 00:54:42 (5936): No heartbeat from core client for 30 sec - exiting 00:54:58 (5936): No heartbeat from core client for 30 sec - exiting 00:55:13 (5936): No heartbeat from core client for 30 sec - exiting 00:55:14 (5936): No heartbeat from core client for 30 sec - exiting 00:55:15 (5936): No heartbeat from core client for 30 sec - exiting 00:55:53 (5936): No heartbeat from core client for 30 sec - exiting 00:55:56 (5936): No heartbeat from core client for 30 sec - exiting 00:56:01 (5936): No heartbeat from core client for 30 sec - exiting 00:56:29 (5936): No heartbeat from core client for 30 sec - exiting 00:56:30 (5936): No heartbeat from core client for 30 sec - exiting 00:56:31 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:02:05 (144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:18:55 (896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4576, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:23:26 (2812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:00:52 (5376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 12:31:21 (1280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:01:16 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:22 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:26:33 (5516): No heartbeat from core client for 30 sec - exiting 13:26:39 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 13:18:33 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:11 (3852): No heartbeat from core client for 30 sec - exiting 13:23:12 (3852): No heartbeat from core client for 30 sec - exiting 13:23:13 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 11:15:00 (4912): No heartbeat from core client for 30 sec - exiting 11:15:01 (4912): No heartbeat from core client for 30 sec - exiting 11:15:02 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:03 (4912): No heartbeat from core client for 30 sec - exiting 11:15:04 (4912): No heartbeat from core client for 30 sec - exiting 11:15:05 (4912): No heartbeat from core client for 30 sec - exiting 11:15:06 (4912): No heartbeat from core client for 30 sec - exiting 11:15:07 (4912): No heartbeat from core client for 30 sec - exiting 11:15:08 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 15:23:03 (5972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:04 (5972): No heartbeat from core client for 30 sec - exiting 15:23:05 (5972): No heartbeat from core client for 30 sec - exiting 18:24:21 (5340): No heartbeat from core client for 30 sec - exiting 18:24:22 (5340): No heartbeat from core client for 30 sec - exiting 18:24:23 (5340): No heartbeat from core client for 30 sec - exiting 18:24:24 (5340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:39 (1432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:13:28 (3668): No heartbeat from core client for 30 sec - exiting 02:13:29 (3668): No heartbeat from core client for 30 sec - exiting 02:13:30 (3668): No heartbeat from core client for 30 sec - exiting 02:13:31 (3668): No heartbeat from core client for 30 sec - exiting 02:13:32 (3668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:52 (1144): No heartbeat from core client for 30 sec - exiting 02:15:53 (1144): No heartbeat from core client for 30 sec - exiting 02:15:54 (1144): No heartbeat from core client for 30 sec - exiting 02:15:55 (1144): No heartbeat from core client for 30 sec - exiting 02:15:56 (1144): No heartbeat from core client for 30 sec - exiting 02:15:57 (1144): No heartbeat from core client for 30 sec - exiting 02:15:58 (1144): No heartbeat from core client for 30 sec - exiting 02:15:59 (1144): No heartbeat from core client for 30 sec - exiting 02:16:00 (1144): No heartbeat from core client for 30 sec - exiting 02:16:01 (1144): No heartbeat from core client for 30 sec - exiting 02:16:02 (1144): No heartbeat from core client for 30 sec - exiting 02:16:03 (1144): No heartbeat from core client for 30 sec - exiting 02:16:04 (1144): No heartbeat from core client for 30 sec - exiting 02:16:05 (1144): No heartbeat from core client for 30 sec - exiting 02:16:06 (1144): No heartbeat from core client for 30 sec - exiting 02:16:07 (1144): No heartbeat from core client for 30 sec - exiting 02:16:08 (1144): No heartbeat from core client for 30 sec - exiting 02:16:09 (1144): No heartbeat from core client for 30 sec - exiting 02:16:10 (1144): No heartbeat from core client for 30 sec - exiting 02:16:11 (1144): No heartbeat from core client for 30 sec - exiting 02:16:12 (1144): No heartbeat from core client for 30 sec - exiting 02:16:13 (1144): No heartbeat from core client for 30 sec - exiting 02:16:14 (1144): No heartbeat from core client for 30 sec - exiting 02:16:15 (1144): No heartbeat from core client for 30 sec - exiting 02:16:16 (1144): No heartbeat from core client for 30 sec - exiting 02:16:17 (1144): No heartbeat from core client for 30 sec - exiting 02:16:18 (1144): No heartbeat from core client for 30 sec - exiting 02:16:19 (1144): No heartbeat from core client for 30 sec - exiting 02:16:20 (1144): No heartbeat from core client for 30 sec - exiting 02:16:21 (1144): No heartbeat from core client for 30 sec - exiting 02:16:22 (1144): No heartbeat from core client for 30 sec - exiting 02:16:23 (1144): No heartbeat from core client for 30 sec - exiting 02:16:24 (1144): No heartbeat from core client for 30 sec - exiting 02:16:25 (1144): No heartbeat from core client for 30 sec - exiting 02:16:26 (1144): No heartbeat from core client for 30 sec - exiting 02:16:27 (1144): No heartbeat from core client for 30 sec - exiting 02:16:28 (1144): No heartbeat from core client for 30 sec - exiting 02:16:29 (1144): No heartbeat from core client for 30 sec - exiting 02:16:30 (1144): No heartbeat from core client for 30 sec - exiting 02:16:31 (1144): No heartbeat from core client for 30 sec - exiting 02:16:32 (1144): No heartbeat from core client for 30 sec - exiting 02:16:33 (1144): No heartbeat from core client for 30 sec - exiting 02:16:34 (1144): No heartbeat from core client for 30 sec - exiting 02:16:35 (1144): No heartbeat from core client for 30 sec - exiting 02:16:36 (1144): No heartbeat from core client for 30 sec - exiting 02:16:37 (1144): No heartbeat from core client for 30 sec - exiting 02:16:38 (1144): No heartbeat from core client for 30 sec - exiting 02:16:39 (1144): No heartbeat from core client for 30 sec - exiting 02:16:40 (1144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 07:27:36 (2128): No heartbeat from core client for 30 sec - exiting 07:27:37 (2128): No heartbeat from core client for 30 sec - exiting 07:27:38 (2128): No heartbeat from core client for 30 sec - exiting 07:27:39 (2128): No heartbeat from core client for 30 sec - exiting 07:27:40 (2128): No heartbeat from core client for 30 sec - exiting 07:27:41 (2128): No heartbeat from core client for 30 sec - exiting 07:27:42 (2128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:50:50 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:18:03 (1280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:28:44 (2116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:13:25 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5908, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish 18:53:17 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 15:22:04 (4860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:12:15 (4224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 16:41:24 (2136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:24 (3804): No heartbeat from core client for 30 sec - exiting 16:43:25 (3804): No heartbeat from core client for 30 sec - exiting 16:43:26 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 13:05:50 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:45 (2576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3160, iMonCtr=1 Model crash detected, will try to restart... 14:47:55 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:04 (3604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5824, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish 14:31:36 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:23 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:57 (4508): No heartbeat from core client for 30 sec - exiting 14:35:58 (4508): No heartbeat from core client for 30 sec - exiting 14:35:59 (4508): No heartbeat from core client for 30 sec - exiting 14:36:00 (4508): No heartbeat from core client for 30 sec - exiting 14:36:01 (4508): No heartbeat from core client for 30 sec - exiting 14:36:02 (4508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:56:27 (4764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3640, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 11:51:27 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 41 - Return code = 16 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 41 - Return code = 16 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 41 - Return code = 16 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 41 - Return code = 16 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 41 - Return code = 16 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 41 - Return code = 16 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Mar 2013 17:01:53	1046143	15650799	hadcm3n_o3d8_2140_40_008282229_2	77,760	107,998	1.3889
22 Mar 2013 09:04:02	1046143	15650799	hadcm3n_o3d8_2140_40_008282229_2	51,840	72,030	1.3895
15 Mar 2013 19:56:38	1046143	15650799	hadcm3n_o3d8_2140_40_008282229_2	25,920	35,998	1.3888