Task 15400739

Name	hadcm3n_o4wp_2060_40_008239535_2
Workunit	8394659
Created	26 Oct 2012, 19:16:00 UTC
Sent	26 Oct 2012, 19:16:04 UTC
Report deadline	26 Jan 2013, 2:43:15 UTC
Received	10 Nov 2012, 0:08:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1239468
Run time	9 days 23 hours 59 min 2 sec
CPU time	9 days 9 hours 55 min 40 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.43 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:33:55 (5473): No heartbeat from core client for 30 sec - exiting 20:36:16 (5473): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:54:19 (5709): No heartbeat from core client for 30 sec - exiting 20:57:22 (5795): Can't acquire lockfile (-154) - waiting 35s 20:59:35 (5841): No heartbeat from core client for 30 sec - exiting 20:59:59 (5841): No heartbeat from core client for 30 sec - exiting 21:00:30 (5841): No heartbeat from core client for 30 sec - exiting 21:00:32 (5841): No heartbeat from core client for 30 sec - exiting 21:00:33 (5841): No heartbeat from core client for 30 sec - exiting 21:00:34 (5841): No heartbeat from core client for 30 sec - exiting 21:00:35 (5841): No heartbeat from core client for 30 sec - exiting 21:00:36 (5841): No heartbeat from core client for 30 sec - exiting 21:00:37 (5841): No heartbeat from core client for 30 sec - exiting 21:00:38 (5841): No heartbeat from core client for 30 sec - exiting 21:00:39 (5841): No heartbeat from core client for 30 sec - exiting 21:00:40 (5841): No heartbeat from core client for 30 sec - exiting 21:00:41 (5841): No heartbeat from core client for 30 sec - exiting 21:00:42 (5841): No heartbeat from core client for 30 sec - exiting 21:00:43 (5841): No heartbeat from core client for 30 sec - exiting 21:03:22 (5943): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:10:54 (6191): Can't acquire lockfile (-154) - waiting 35s CPDN Monitor - Quit request from BOINC... 21:13:24 (6245): No heartbeat from core client for 30 sec - exiting 21:13:25 (6245): No heartbeat from core client for 30 sec - exiting 21:13:26 (6245): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:18:25 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6346, selfPID=6346, iMonCtr=1 21:39:36 (6378): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:35:09 (6524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:35:11 (6524): No heartbeat from core client for 30 sec - exiting 22:50:56 (6561): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:07:25 (6607): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:07:27 (6607): No heartbeat from core client for 30 sec - exiting 23:07:28 (6607): No heartbeat from core client for 30 sec - exiting 23:14:33 (6661): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:20:17 (6724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:20:18 (6724): No heartbeat from core client for 30 sec - exiting 23:20:19 (6724): No heartbeat from core client for 30 sec - exiting 23:20:20 (6724): No heartbeat from core client for 30 sec - exiting 23:20:21 (6724): No heartbeat from core client for 30 sec - exiting 23:24:40 (6752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:26:46 (6752): No heartbeat from core client for 30 sec - exiting 23:26:47 (6752): No heartbeat from core client for 30 sec - exiting 23:26:48 (6752): No heartbeat from core client for 30 sec - exiting 23:26:49 (6752): No heartbeat from core client for 30 sec - exiting 23:26:50 (6752): No heartbeat from core client for 30 sec - exiting 23:26:51 (6752): No heartbeat from core client for 30 sec - exiting 23:26:52 (6752): No heartbeat from core client for 30 sec - exiting 23:26:53 (6752): No heartbeat from core client for 30 sec - exiting 23:26:54 (6752): No heartbeat from core client for 30 sec - exiting 23:26:55 (6752): No heartbeat from core client for 30 sec - exiting 23:26:56 (6752): No heartbeat from core client for 30 sec - exiting 23:26:57 (6752): No heartbeat from core client for 30 sec - exiting 23:26:58 (6752): No heartbeat from core client for 30 sec - exiting 23:26:59 (6752): No heartbeat from core client for 30 sec - exiting 23:27:00 (6752): No heartbeat from core client for 30 sec - exiting 23:27:01 (6752): No heartbeat from core client for 30 sec - exiting 23:27:02 (6752): No heartbeat from core client for 30 sec - exiting 23:27:03 (6752): No heartbeat from core client for 30 sec - exiting 23:27:04 (6752): No heartbeat from core client for 30 sec - exiting 23:27:05 (6752): No heartbeat from core client for 30 sec - exiting 23:27:06 (6752): No heartbeat from core client for 30 sec - exiting 23:27:07 (6752): No heartbeat from core client for 30 sec - exiting 23:27:08 (6752): No heartbeat from core client for 30 sec - exiting 23:27:09 (6752): No heartbeat from core client for 30 sec - exiting 23:27:10 (6752): No heartbeat from core client for 30 sec - exiting 23:27:11 (6752): No heartbeat from core client for 30 sec - exiting 23:27:12 (6752): No heartbeat from core client for 30 sec - exiting 23:27:13 (6752): No heartbeat from core client for 30 sec - exiting 23:27:14 (6752): No heartbeat from core client for 30 sec - exiting 23:27:15 (6752): No heartbeat from core client for 30 sec - exiting 23:27:16 (6752): No heartbeat from core client for 30 sec - exiting 23:27:17 (6752): No heartbeat from core client for 30 sec - exiting 23:27:18 (6752): No heartbeat from core client for 30 sec - exiting 23:27:19 (6752): No heartbeat from core client for 30 sec - exiting 23:27:20 (6752): No heartbeat from core client for 30 sec - exiting 23:27:21 (6752): No heartbeat from core client for 30 sec - exiting 23:27:22 (6752): No heartbeat from core client for 30 sec - exiting 23:27:23 (6752): No heartbeat from core client for 30 sec - exiting 23:27:24 (6752): No heartbeat from core client for 30 sec - exiting 23:27:25 (6752): No heartbeat from core client for 30 sec - exiting 23:27:26 (6752): No heartbeat from core client for 30 sec - exiting 23:27:27 (6752): No heartbeat from core client for 30 sec - exiting 23:27:28 (6752): No heartbeat from core client for 30 sec - exiting 23:27:29 (6752): No heartbeat from core client for 30 sec - exiting 23:27:30 (6752): No heartbeat from core client for 30 sec - exiting 23:27:31 (6752): No heartbeat from core client for 30 sec - exiting 23:27:32 (6752): No heartbeat from core client for 30 sec - exiting 23:27:33 (6752): No heartbeat from core client for 30 sec - exiting 23:27:34 (6752): No heartbeat from core client for 30 sec - exiting 23:27:35 (6752): No heartbeat from core client for 30 sec - exiting 23:27:36 (6752): No heartbeat from core client for 30 sec - exiting 23:27:37 (6752): No heartbeat from core client for 30 sec - exiting 23:27:38 (6752): No heartbeat from core client for 30 sec - exiting 23:27:39 (6752): No heartbeat from core client for 30 sec - exiting 23:27:40 (6752): No heartbeat from core client for 30 sec - exiting 23:36:53 (6817): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:37:43 (6865): No heartbeat from core client for 30 sec - exiting 23:37:44 (6865): No heartbeat from core client for 30 sec - exiting 23:37:45 (6865): No heartbeat from core client for 30 sec - exiting 23:37:46 (6865): No heartbeat from core client for 30 sec - exiting 23:37:47 (6865): No heartbeat from core client for 30 sec - exiting 23:37:55 (6865): No heartbeat from core client for 30 sec - exiting 23:37:56 (6865): No heartbeat from core client for 30 sec - exiting 23:37:57 (6865): No heartbeat from core client for 30 sec - exiting 23:37:58 (6865): No heartbeat from core client for 30 sec - exiting 23:37:59 (6865): No heartbeat from core client for 30 sec - exiting 23:38:00 (6865): No heartbeat from core client for 30 sec - exiting 23:38:01 (6865): No heartbeat from core client for 30 sec - exiting 23:38:02 (6865): No heartbeat from core client for 30 sec - exiting 23:38:03 (6865): No heartbeat from core client for 30 sec - exiting 23:38:04 (6865): No heartbeat from core client for 30 sec - exiting 23:38:05 (6865): No heartbeat from core client for 30 sec - exiting 23:38:06 (6865): No heartbeat from core client for 30 sec - exiting 23:38:07 (6865): No heartbeat from core client for 30 sec - exiting 23:38:08 (6865): No heartbeat from core client for 30 sec - exiting 23:38:09 (6865): No heartbeat from core client for 30 sec - exiting 23:38:10 (6865): No heartbeat from core client for 30 sec - exiting 23:38:12 (6865): No heartbeat from core client for 30 sec - exiting 23:38:13 (6865): No heartbeat from core client for 30 sec - exiting 23:38:14 (6865): No heartbeat from core client for 30 sec - exiting 23:38:15 (6865): No heartbeat from core client for 30 sec - exiting 23:38:16 (6865): No heartbeat from core client for 30 sec - exiting 23:38:18 (6865): No heartbeat from core client for 30 sec - exiting 23:38:19 (6865): No heartbeat from core client for 30 sec - exiting 23:38:20 (6865): No heartbeat from core client for 30 sec - exiting 23:38:21 (6865): No heartbeat from core client for 30 sec - exiting 23:38:22 (6865): No heartbeat from core client for 30 sec - exiting 23:38:23 (6865): No heartbeat from core client for 30 sec - exiting 23:38:24 (6865): No heartbeat from core client for 30 sec - exiting 23:38:25 (6865): No heartbeat from core client for 30 sec - exiting 23:38:26 (6865): No heartbeat from core client for 30 sec - exiting 23:38:27 (6865): No heartbeat from core client for 30 sec - exiting 23:38:28 (6865): No heartbeat from core client for 30 sec - exiting 23:38:29 (6865): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:26 (6883): No heartbeat from core client for 30 sec - exiting 23:40:27 (6883): No heartbeat from core client for 30 sec - exiting 23:40:28 (6883): No heartbeat from core client for 30 sec - exiting 23:40:29 (6883): No heartbeat from core client for 30 sec - exiting 23:40:30 (6883): No heartbeat from core client for 30 sec - exiting 23:40:35 (6883): No heartbeat from core client for 30 sec - exiting 23:40:36 (6883): No heartbeat from core client for 30 sec - exiting 23:40:37 (6883): No heartbeat from core client for 30 sec - exiting 23:40:38 (6883): No heartbeat from core client for 30 sec - exiting 23:40:40 (6883): No heartbeat from core client for 30 sec - exiting 23:40:41 (6883): No heartbeat from core client for 30 sec - exiting 23:40:42 (6883): No heartbeat from core client for 30 sec - exiting 23:40:43 (6883): No heartbeat from core client for 30 sec - exiting 23:40:44 (6883): No heartbeat from core client for 30 sec - exiting 23:40:45 (6883): No heartbeat from core client for 30 sec - exiting 23:41:17 (6883): No heartbeat from core client for 30 sec - exiting 23:41:18 (6883): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:42 (6944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:43 (6944): No heartbeat from core client for 30 sec - exiting 23:55:02 (6944): No heartbeat from core client for 30 sec - exiting 23:55:03 (6944): No heartbeat from core client for 30 sec - exiting 23:55:04 (6944): No heartbeat from core client for 30 sec - exiting 23:55:05 (6944): No heartbeat from core client for 30 sec - exiting 23:55:06 (6944): No heartbeat from core client for 30 sec - exiting 23:55:07 (6944): No heartbeat from core client for 30 sec - exiting 23:55:08 (6944): No heartbeat from core client for 30 sec - exiting 23:55:09 (6944): No heartbeat from core client for 30 sec - exiting 23:55:10 (6944): No heartbeat from core client for 30 sec - exiting 23:55:11 (6944): No heartbeat from core client for 30 sec - exiting 23:55:12 (6944): No heartbeat from core client for 30 sec - exiting 23:55:13 (6944): No heartbeat from core client for 30 sec - exiting 23:55:14 (6944): No heartbeat from core client for 30 sec - exiting 23:55:15 (6944): No heartbeat from core client for 30 sec - exiting 23:55:16 (6944): No heartbeat from core client for 30 sec - exiting 23:55:17 (6944): No heartbeat from core client for 30 sec - exiting 23:55:18 (6944): No heartbeat from core client for 30 sec - exiting 23:55:19 (6944): No heartbeat from core client for 30 sec - exiting 00:02:09 (7025): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:04:08 (7054): No heartbeat from core client for 30 sec - exiting 00:04:09 (7054): No heartbeat from core client for 30 sec - exiting 00:04:10 (7054): No heartbeat from core client for 30 sec - exiting 00:04:11 (7054): No heartbeat from core client for 30 sec - exiting 00:04:12 (7054): No heartbeat from core client for 30 sec - exiting 00:04:13 (7054): No heartbeat from core client for 30 sec - exiting 00:04:14 (7054): No heartbeat from core client for 30 sec - exiting 00:04:15 (7054): No heartbeat from core client for 30 sec - exiting 00:04:16 (7054): No heartbeat from core client for 30 sec - exiting 00:04:17 (7054): No heartbeat from core client for 30 sec - exiting 00:04:18 (7054): No heartbeat from core client for 30 sec - exiting 00:04:51 (7054): No heartbeat from core client for 30 sec - exiting 00:04:52 (7054): No heartbeat from core client for 30 sec - exiting 00:04:53 (7054): No heartbeat from core client for 30 sec - exiting 00:04:54 (7054): No heartbeat from core client for 30 sec - exiting 00:04:55 (7054): No heartbeat from core client for 30 sec - exiting 00:04:56 (7054): No heartbeat from core client for 30 sec - exiting 00:04:57 (7054): No heartbeat from core client for 30 sec - exiting 00:04:58 (7054): No heartbeat from core client for 30 sec - exiting 00:04:59 (7054): No heartbeat from core client for 30 sec - exiting 00:05:00 (7054): No heartbeat from core client for 30 sec - exiting 00:05:01 (7054): No heartbeat from core client for 30 sec - exiting 00:05:02 (7054): No heartbeat from core client for 30 sec - exiting 00:05:03 (7054): No heartbeat from core client for 30 sec - exiting 00:05:05 (7054): No heartbeat from core client for 30 sec - exiting 00:05:06 (7054): No heartbeat from core client for 30 sec - exiting 00:05:07 (7054): No heartbeat from core client for 30 sec - exiting 00:05:08 (7054): No heartbeat from core client for 30 sec - exiting 00:05:09 (7054): No heartbeat from core client for 30 sec - exiting 00:05:10 (7054): No heartbeat from core client for 30 sec - exiting 00:05:11 (7054): No heartbeat from core client for 30 sec - exiting 00:05:12 (7054): No heartbeat from core client for 30 sec - exiting 00:05:13 (7054): No heartbeat from core client for 30 sec - exiting 00:05:14 (7054): No heartbeat from core client for 30 sec - exiting 00:05:15 (7054): No heartbeat from core client for 30 sec - exiting 00:05:16 (7054): No heartbeat from core client for 30 sec - exiting 00:05:17 (7054): No heartbeat from core client for 30 sec - exiting 00:05:18 (7054): No heartbeat from core client for 30 sec - exiting 00:05:19 (7054): No heartbeat from core client for 30 sec - exiting 00:05:20 (7054): No heartbeat from core client for 30 sec - exiting 00:05:21 (7054): No heartbeat from core client for 30 sec - exiting 00:05:22 (7054): No heartbeat from core client for 30 sec - exiting 00:05:23 (7080): Can't acquire lockfile (-154) - waiting 35s 00:06:42 (7080): No heartbeat from core client for 30 sec - exiting 00:06:50 (7080): No heartbeat from core client for 30 sec - exiting 00:06:51 (7080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:22 (7103): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:11:35 (7125): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:11:47 (7125): No heartbeat from core client for 30 sec - exiting 00:11:48 (7125): No heartbeat from core client for 30 sec - exiting 00:12:55 (7166): No heartbeat from core client for 30 sec - exiting 00:13:26 (7166): No heartbeat from core client for 30 sec - exiting 00:13:27 (7166): No heartbeat from core client for 30 sec - exiting 00:13:28 (7166): No heartbeat from core client for 30 sec - exiting 00:13:29 (7166): No heartbeat from core client for 30 sec - exiting 00:13:30 (7166): No heartbeat from core client for 30 sec - exiting 00:13:31 (7166): No heartbeat from core client for 30 sec - exiting 00:13:43 (7166): No heartbeat from core client for 30 sec - exiting 00:14:48 (7166): No heartbeat from core client for 30 sec - exiting 00:14:49 (7166): No heartbeat from core client for 30 sec - exiting 00:14:50 (7166): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:15:40 (7166): No heartbeat from core client for 30 sec - exiting 00:22:22 (7208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:23 (7208): No heartbeat from core client for 30 sec - exiting 00:22:24 (7208): No heartbeat from core client for 30 sec - exiting 00:22:25 (7208): No heartbeat from core client for 30 sec - exiting 00:22:26 (7208): No heartbeat from core client for 30 sec - exiting 00:22:27 (7208): No heartbeat from core client for 30 sec - exiting 00:22:28 (7208): No heartbeat from core client for 30 sec - exiting 00:22:29 (7208): No heartbeat from core client for 30 sec - exiting 00:22:30 (7208): No heartbeat from core client for 30 sec - exiting 00:22:31 (7208): No heartbeat from core client for 30 sec - exiting 00:22:32 (7208): No heartbeat from core client for 30 sec - exiting 00:22:33 (7208): No heartbeat from core client for 30 sec - exiting 00:22:34 (7208): No heartbeat from core client for 30 sec - exiting 00:22:35 (7208): No heartbeat from core client for 30 sec - exiting 00:22:36 (7208): No heartbeat from core client for 30 sec - exiting 00:22:37 (7208): No heartbeat from core client for 30 sec - exiting 00:22:38 (7208): No heartbeat from core client for 30 sec - exiting 00:22:39 (7208): No heartbeat from core client for 30 sec - exiting 00:22:40 (7208): No heartbeat from core client for 30 sec - exiting 00:23:12 (7208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 00:29:37 (7310): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:33:20 (7328): No heartbeat from core client for 30 sec - exiting 00:33:21 (7328): No heartbeat from core client for 30 sec - exiting 00:33:22 (7328): No heartbeat from core client for 30 sec - exiting 00:33:30 (7328): No heartbeat from core client for 30 sec - exiting 00:33:32 (7328): No heartbeat from core client for 30 sec - exiting 00:33:33 (7328): No heartbeat from core client for 30 sec - exiting 00:33:34 (7328): No heartbeat from core client for 30 sec - exiting 00:33:35 (7328): No heartbeat from core client for 30 sec - exiting 00:33:36 (7328): No heartbeat from core client for 30 sec - exiting 00:33:37 (7328): No heartbeat from core client for 30 sec - exiting 00:33:38 (7328): No heartbeat from core client for 30 sec - exiting 00:33:39 (7328): No heartbeat from core client for 30 sec - exiting 00:33:40 (7328): No heartbeat from core client for 30 sec - exiting 00:33:42 (7328): No heartbeat from core client for 30 sec - exiting 00:33:43 (7328): No heartbeat from core client for 30 sec - exiting 00:33:44 (7328): No heartbeat from core client for 30 sec - exiting 00:33:45 (7328): No heartbeat from core client for 30 sec - exiting 00:33:46 (7328): No heartbeat from core client for 30 sec - exiting 00:34:48 (7328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:40:00 (7373): No heartbeat from core client for 30 sec - exiting 00:40:51 (7373): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:46:46 (7416): No heartbeat from core client for 30 sec - exiting 00:46:48 (7416): No heartbeat from core client for 30 sec - exiting 00:46:49 (7416): No heartbeat from core client for 30 sec - exiting 00:46:50 (7416): No heartbeat from core client for 30 sec - exiting 00:46:51 (7416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:03 (7416): No heartbeat from core client for 30 sec - exiting 00:49:02 (7457): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:06:21 (7517): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:06:23 (7517): No heartbeat from core client for 30 sec - exiting 01:06:24 (7517): No heartbeat from core client for 30 sec - exiting 01:06:25 (7517): No heartbeat from core client for 30 sec - exiting 01:06:26 (7517): No heartbeat from core client for 30 sec - exiting 01:06:27 (7517): No heartbeat from core client for 30 sec - exiting 01:06:28 (7517): No heartbeat from core client for 30 sec - exiting 01:06:29 (7517): No heartbeat from core client for 30 sec - exiting 01:06:30 (7517): No heartbeat from core client for 30 sec - exiting 01:06:31 (7517): No heartbeat from core client for 30 sec - exiting 01:06:32 (7517): No heartbeat from core client for 30 sec - exiting 01:06:33 (7517): No heartbeat from core client for 30 sec - exiting 01:06:34 (7517): No heartbeat from core client for 30 sec - exiting 01:06:35 (7517): No heartbeat from core client for 30 sec - exiting 01:06:36 (7517): No heartbeat from core client for 30 sec - exiting 01:06:37 (7517): No heartbeat from core client for 30 sec - exiting 01:06:38 (7517): No heartbeat from core client for 30 sec - exiting 01:06:39 (7517): No heartbeat from core client for 30 sec - exiting 01:06:40 (7517): No heartbeat from core client for 30 sec - exiting 01:06:41 (7517): No heartbeat from core client for 30 sec - exiting 01:06:42 (7517): No heartbeat from core client for 30 sec - exiting 01:06:43 (7517): No heartbeat from core client for 30 sec - exiting 01:06:44 (7517): No heartbeat from core client for 30 sec - exiting 01:06:45 (7517): No heartbeat from core client for 30 sec - exiting 01:06:46 (7517): No heartbeat from core client for 30 sec - exiting 01:06:47 (7517): No heartbeat from core client for 30 sec - exiting 01:06:48 (7517): No heartbeat from core client for 30 sec - exiting 01:06:49 (7517): No heartbeat from core client for 30 sec - exiting 01:06:50 (7517): No heartbeat from core client for 30 sec - exiting 01:06:51 (7517): No heartbeat from core client for 30 sec - exiting 01:06:52 (7517): No heartbeat from core client for 30 sec - exiting 01:06:53 (7517): No heartbeat from core client for 30 sec - exiting 01:06:54 (7517): No heartbeat from core client for 30 sec - exiting 01:06:55 (7517): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:54:34 (28991): Can't acquire lockfile (-154) - waiting 35s 20:55:10 (28991): Can't acquire lockfile (-154) - exiting 21:02:35 (29092): Can't acquire lockfile (-154) - waiting 35s 21:03:10 (29092): Can't acquire lockfile (-154) - exiting 21:07:47 (29119): No heartbeat from core client for 30 sec - exiting 21:12:32 (29119): No heartbeat from core client for 30 sec - exiting 21:12:33 (29119): No heartbeat from core client for 30 sec - exiting 21:12:34 (29119): No heartbeat from core client for 30 sec - exiting 21:12:35 (29119): No heartbeat from core client for 30 sec - exiting 21:12:36 (29119): No heartbeat from core client for 30 sec - exiting 21:12:37 (29119): No heartbeat from core client for 30 sec - exiting 21:12:38 (29119): No heartbeat from core client for 30 sec - exiting 21:12:39 (29119): No heartbeat from core client for 30 sec - exiting 21:12:40 (29119): No heartbeat from core client for 30 sec - exiting 21:12:41 (29119): No heartbeat from core client for 30 sec - exiting 21:12:42 (29119): No heartbeat from core client for 30 sec - exiting 21:12:43 (29119): No heartbeat from core client for 30 sec - exiting 21:12:44 (29119): No heartbeat from core client for 30 sec - exiting 21:12:45 (29119): No heartbeat from core client for 30 sec - exiting 21:12:46 (29119): No heartbeat from core client for 30 sec - exiting 21:12:47 (29119): No heartbeat from core client for 30 sec - exiting 21:12:48 (29119): No heartbeat from core client for 30 sec - exiting 21:12:49 (29119): No heartbeat from core client for 30 sec - exiting 21:12:50 (29119): No heartbeat from core client for 30 sec - exiting 21:12:51 (29119): No heartbeat from core client for 30 sec - exiting 21:12:53 (29119): No heartbeat from core client for 30 sec - exiting 21:12:54 (29119): No heartbeat from core client for 30 sec - exiting 21:12:55 (29119): No heartbeat from core client for 30 sec - exiting 21:12:56 (29119): No heartbeat from core client for 30 sec - exiting 21:12:57 (29119): No heartbeat from core client for 30 sec - exiting 21:12:58 (29119): No heartbeat from core client for 30 sec - exiting 21:12:59 (29119): No heartbeat from core client for 30 sec - exiting 21:13:00 (29119): No heartbeat from core client for 30 sec - exiting 21:13:01 (29119): No heartbeat from core client for 30 sec - exiting 21:13:02 (29119): No heartbeat from core client for 30 sec - exiting 21:13:03 (29119): No heartbeat from core client for 30 sec - exiting 21:13:04 (29119): No heartbeat from core client for 30 sec - exiting 21:13:05 (29119): No heartbeat from core client for 30 sec - exiting 21:13:06 (29119): No heartbeat from core client for 30 sec - exiting 21:13:07 (29119): No heartbeat from core client for 30 sec - exiting 21:13:08 (29119): No heartbeat from core client for 30 sec - exiting 21:14:29 (29119): No heartbeat from core client for 30 sec - exiting 21:14:30 (29119): No heartbeat from core client for 30 sec - exiting 21:14:31 (29119): No heartbeat from core client for 30 sec - exiting 21:14:32 (29119): No heartbeat from core client for 30 sec - exiting 21:14:33 (29119): No heartbeat from core client for 30 sec - exiting 21:14:34 (29119): No heartbeat from core client for 30 sec - exiting 21:14:35 (29119): No heartbeat from core client for 30 sec - exiting 21:14:36 (29119): No heartbeat from core client for 30 sec - exiting 21:14:37 (29119): No heartbeat from core client for 30 sec - exiting 21:14:38 (29119): No heartbeat from core client for 30 sec - exiting 21:14:39 (29119): No heartbeat from core client for 30 sec - exiting 21:14:40 (29119): No heartbeat from core client for 30 sec - exiting 23:50:34 (30211): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:55 (30480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/ocean_restart.day after 11 attempts CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /mnt/users1/staf/wes/BOINC/BOINC/projects/climateprediction.net/hadcm3n_o4wp_2060_40_008239535/dataout/ocean_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Nov 2012 00:09:41	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	259,200	813,356	3.1379
08 Nov 2012 02:13:37	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	233,280	730,768	3.1326
06 Nov 2012 05:00:12	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	207,360	654,193	3.1549
04 Nov 2012 14:01:20	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	181,440	574,449	3.1661
03 Nov 2012 17:05:54	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	155,520	496,728	3.1940
02 Nov 2012 07:29:45	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	129,600	420,838	3.2472
31 Oct 2012 08:23:45	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	103,680	343,309	3.3112
30 Oct 2012 08:49:58	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	77,760	262,113	3.3708
29 Oct 2012 03:46:01	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	51,840	176,240	3.3997
28 Oct 2012 01:48:13	1239468	15400739	hadcm3n_o4wp_2060_40_008239535_2	25,920	85,865	3.3127