Task 15500392

Name	hadcm3n_3mf1_1940_40_008264350_1
Workunit	8419474
Created	23 Dec 2012, 6:00:56 UTC
Sent	23 Dec 2012, 6:19:05 UTC
Report deadline	24 Mar 2013, 13:46:16 UTC
Received	3 Jan 2013, 23:08:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1087577
Run time	5 days 16 hours 53 min 50 sec
CPU time	5 days 7 hours 0 min 52 sec
Validate state	Invalid
Credit	1,555.20
Device peak FLOPS	1.65 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:51:54 (6596): No heartbeat from core client for 30 sec - exiting 15:51:55 (6596): No heartbeat from core client for 30 sec - exiting 15:51:56 (6596): No heartbeat from core client for 30 sec - exiting 15:51:57 (6596): No heartbeat from core client for 30 sec - exiting 15:51:58 (6596): No heartbeat from core client for 30 sec - exiting 15:51:59 (6596): No heartbeat from core client for 30 sec - exiting 15:52:00 (6596): No heartbeat from core client for 30 sec - exiting 15:52:01 (6596): No heartbeat from core client for 30 sec - exiting 15:52:02 (6596): No heartbeat from core client for 30 sec - exiting 15:52:03 (6596): No heartbeat from core client for 30 sec - exiting 15:52:04 (6596): No heartbeat from core client for 30 sec - exiting 15:52:05 (6596): No heartbeat from core client for 30 sec - exiting 15:52:06 (6596): No heartbeat from core client for 30 sec - exiting 15:52:07 (6596): No heartbeat from core client for 30 sec - exiting 15:52:08 (6596): No heartbeat from core client for 30 sec - exiting 15:52:09 (6596): No heartbeat from core client for 30 sec - exiting 15:52:10 (6596): No heartbeat from core client for 30 sec - exiting 15:52:11 (6596): No heartbeat from core client for 30 sec - exiting 15:52:12 (6596): No heartbeat from core client for 30 sec - exiting 15:52:13 (6596): No heartbeat from core client for 30 sec - exiting 15:52:14 (6596): No heartbeat from core client for 30 sec - exiting 15:52:15 (6596): No heartbeat from core client for 30 sec - exiting 15:52:16 (6596): No heartbeat from core client for 30 sec - exiting 15:52:17 (6596): No heartbeat from core client for 30 sec - exiting 15:52:18 (6596): No heartbeat from core client for 30 sec - exiting 15:52:19 (6596): No heartbeat from core client for 30 sec - exiting 15:52:20 (6596): No heartbeat from core client for 30 sec - exiting 15:52:21 (6596): No heartbeat from core client for 30 sec - exiting 15:52:22 (6596): No heartbeat from core client for 30 sec - exiting 15:52:23 (6596): No heartbeat from core client for 30 sec - exiting 15:52:24 (6596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:19 (6992): No heartbeat from core client for 30 sec - exiting 17:31:20 (6992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:21 (6992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:48:53 (7644): No heartbeat from core client for 30 sec - exiting 17:48:54 (7644): No heartbeat from core client for 30 sec - exiting 17:48:55 (7644): No heartbeat from core client for 30 sec - exiting 17:48:56 (7644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:48:57 (7644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:42:06 (1452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:15:41 (7836): No heartbeat from core client for 30 sec - exiting 04:15:42 (7836): No heartbeat from core client for 30 sec - exiting 04:15:43 (7836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:44 (7836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7900, iMonCtr=1 Model crash detected, will try to restart... 10:57:22 (7816): No heartbeat from core client for 30 sec - exiting 10:57:23 (7816): No heartbeat from core client for 30 sec - exiting 10:57:24 (7816): No heartbeat from core client for 30 sec - exiting 10:57:25 (7816): No heartbeat from core client for 30 sec - exiting 10:57:26 (7816): No heartbeat from core client for 30 sec - exiting 10:57:27 (7816): No heartbeat from core client for 30 sec - exiting 10:57:28 (7816): No heartbeat from core client for 30 sec - exiting 10:57:29 (7816): No heartbeat from core client for 30 sec - exiting 10:57:30 (7816): No heartbeat from core client for 30 sec - exiting 10:57:31 (7816): No heartbeat from core client for 30 sec - exiting 10:57:32 (7816): No heartbeat from core client for 30 sec - exiting 10:57:33 (7816): No heartbeat from core client for 30 sec - exiting 10:57:34 (7816): No heartbeat from core client for 30 sec - exiting 10:57:35 (7816): No heartbeat from core client for 30 sec - exiting 10:57:36 (7816): No heartbeat from core client for 30 sec - exiting 10:57:37 (7816): No heartbeat from core client for 30 sec - exiting 10:57:38 (7816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:43:03 (8376): No heartbeat from core client for 30 sec - exiting 15:43:04 (8376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:50:56 (3564): No heartbeat from core client for 30 sec - exiting 14:50:57 (3564): No heartbeat from core client for 30 sec - exiting 14:50:58 (3564): No heartbeat from core client for 30 sec - exiting 14:50:59 (3564): No heartbeat from core client for 30 sec - exiting 14:51:00 (3564): No heartbeat from core client for 30 sec - exiting 14:51:01 (3564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:46:08 (11020): No heartbeat from core client for 30 sec - exiting 17:46:10 (11020): No heartbeat from core client for 30 sec - exiting 17:46:11 (11020): No heartbeat from core client for 30 sec - exiting 17:46:12 (11020): No heartbeat from core client for 30 sec - exiting 17:46:13 (11020): No heartbeat from core client for 30 sec - exiting 17:46:14 (11020): No heartbeat from core client for 30 sec - exiting 17:46:15 (11020): No heartbeat from core client for 30 sec - exiting 17:46:16 (11020): No heartbeat from core client for 30 sec - exiting 17:46:17 (11020): No heartbeat from core client for 30 sec - exiting 17:46:18 (11020): No heartbeat from core client for 30 sec - exiting 17:46:19 (11020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:12:10 (4260): No heartbeat from core client for 30 sec - exiting 17:12:11 (4260): No heartbeat from core client for 30 sec - exiting 17:12:12 (4260): No heartbeat from core client for 30 sec - exiting 17:12:13 (4260): No heartbeat from core client for 30 sec - exiting 17:12:14 (4260): No heartbeat from core client for 30 sec - exiting 17:12:15 (4260): No heartbeat from core client for 30 sec - exiting 17:12:16 (4260): No heartbeat from core client for 30 sec - exiting 17:12:17 (4260): No heartbeat from core client for 30 sec - exiting 17:12:18 (4260): No heartbeat from core client for 30 sec - exiting 17:12:19 (4260): No heartbeat from core client for 30 sec - exiting 17:12:20 (4260): No heartbeat from core client for 30 sec - exiting 17:12:21 (4260): No heartbeat from core client for 30 sec - exiting 17:12:22 (4260): No heartbeat from core client for 30 sec - exiting 17:12:23 (4260): No heartbeat from core client for 30 sec - exiting 17:12:24 (4260): No heartbeat from core client for 30 sec - exiting 17:12:25 (4260): No heartbeat from core client for 30 sec - exiting 17:12:26 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:14:12 (6320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:28:01 (5824): No heartbeat from core client for 30 sec - exiting 19:28:03 (5824): No heartbeat from core client for 30 sec - exiting 19:28:04 (5824): No heartbeat from core client for 30 sec - exiting 19:28:05 (5824): No heartbeat from core client for 30 sec - exiting 19:28:06 (5824): No heartbeat from core client for 30 sec - exiting 19:28:07 (5824): No heartbeat from core client for 30 sec - exiting 19:28:08 (5824): No heartbeat from core client for 30 sec - exiting 19:28:09 (5824): No heartbeat from core client for 30 sec - exiting 19:28:10 (5824): No heartbeat from core client for 30 sec - exiting 19:28:11 (5824): No heartbeat from core client for 30 sec - exiting 19:28:12 (5824): No heartbeat from core client for 30 sec - exiting 19:28:13 (5824): No heartbeat from core client for 30 sec - exiting 19:28:14 (5824): No heartbeat from core client for 30 sec - exiting 19:28:15 (5824): No heartbeat from core client for 30 sec - exiting 19:28:16 (5824): No heartbeat from core client for 30 sec - exiting 19:28:17 (5824): No heartbeat from core client for 30 sec - exiting 19:28:18 (5824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:28:19 (5824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:11:34 (7996): No heartbeat from core client for 30 sec - exiting 21:11:35 (7996): No heartbeat from core client for 30 sec - exiting 21:11:36 (7996): No heartbeat from core client for 30 sec - exiting 21:11:37 (7996): No heartbeat from core client for 30 sec - exiting 21:11:38 (7996): No heartbeat from core client for 30 sec - exiting 21:11:39 (7996): No heartbeat from core client for 30 sec - exiting 21:11:40 (7996): No heartbeat from core client for 30 sec - exiting 21:11:41 (7996): No heartbeat from core client for 30 sec - exiting 21:11:42 (7996): No heartbeat from core client for 30 sec - exiting 21:11:43 (7996): No heartbeat from core client for 30 sec - exiting 21:11:44 (7996): No heartbeat from core client for 30 sec - exiting 21:11:45 (7996): No heartbeat from core client for 30 sec - exiting 21:11:46 (7996): No heartbeat from core client for 30 sec - exiting 21:11:47 (7996): No heartbeat from core client for 30 sec - exiting 21:11:48 (7996): No heartbeat from core client for 30 sec - exiting 21:11:49 (7996): No heartbeat from core client for 30 sec - exiting 21:11:50 (7996): No heartbeat from core client for 30 sec - exiting 21:11:51 (7996): No heartbeat from core client for 30 sec - exiting 21:11:52 (7996): No heartbeat from core client for 30 sec - exiting 21:11:53 (7996): No heartbeat from core client for 30 sec - exiting 21:11:54 (7996): No heartbeat from core client for 30 sec - exiting 21:11:55 (7996): No heartbeat from core client for 30 sec - exiting 21:11:56 (7996): No heartbeat from core client for 30 sec - exiting 21:11:57 (7996): No heartbeat from core client for 30 sec - exiting 21:11:58 (7996): No heartbeat from core client for 30 sec - exiting 21:11:59 (7996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:05:34 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:33:22 (8012): No heartbeat from core client for 30 sec - exiting 17:33:23 (8012): No heartbeat from core client for 30 sec - exiting 17:33:24 (8012): No heartbeat from core client for 30 sec - exiting 17:33:25 (8012): No heartbeat from core client for 30 sec - exiting 17:33:26 (8012): No heartbeat from core client for 30 sec - exiting 17:33:27 (8012): No heartbeat from core client for 30 sec - exiting 17:33:28 (8012): No heartbeat from core client for 30 sec - exiting 17:33:29 (8012): No heartbeat from core client for 30 sec - exiting 17:33:30 (8012): No heartbeat from core client for 30 sec - exiting 17:33:31 (8012): No heartbeat from core client for 30 sec - exiting 17:33:32 (8012): No heartbeat from core client for 30 sec - exiting 17:33:33 (8012): No heartbeat from core client for 30 sec - exiting 17:33:34 (8012): No heartbeat from core client for 30 sec - exiting 17:33:35 (8012): No heartbeat from core client for 30 sec - exiting 17:33:36 (8012): No heartbeat from core client for 30 sec - exiting 17:33:37 (8012): No heartbeat from core client for 30 sec - exiting 17:33:38 (8012): No heartbeat from core client for 30 sec - exiting 17:33:39 (8012): No heartbeat from core client for 30 sec - exiting 17:33:40 (8012): No heartbeat from core client for 30 sec - exiting 17:33:41 (8012): No heartbeat from core client for 30 sec - exiting 17:33:42 (8012): No heartbeat from core client for 30 sec - exiting 17:33:43 (8012): No heartbeat from core client for 30 sec - exiting 17:33:44 (8012): No heartbeat from core client for 30 sec - exiting 17:33:45 (8012): No heartbeat from core client for 30 sec - exiting 17:33:46 (8012): No heartbeat from core client for 30 sec - exiting 17:33:47 (8012): No heartbeat from core client for 30 sec - exiting 17:33:48 (8012): No heartbeat from core client for 30 sec - exiting 17:33:49 (8012): No heartbeat from core client for 30 sec - exiting 17:33:50 (8012): No heartbeat from core client for 30 sec - exiting 17:33:51 (8012): No heartbeat from core client for 30 sec - exiting 17:33:52 (8012): No heartbeat from core client for 30 sec - exiting 17:33:53 (8012): No heartbeat from core client for 30 sec - exiting 17:33:54 (8012): No heartbeat from core client for 30 sec - exiting 17:33:55 (8012): No heartbeat from core client for 30 sec - exiting 17:33:56 (8012): No heartbeat from core client for 30 sec - exiting 17:33:57 (8012): No heartbeat from core client for 30 sec - exiting 17:33:58 (8012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:59 (8012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:13:24 (7968): No heartbeat from core client for 30 sec - exiting 05:13:26 (7968): No heartbeat from core client for 30 sec - exiting 05:13:27 (7968): No heartbeat from core client for 30 sec - exiting 05:13:28 (7968): No heartbeat from core client for 30 sec - exiting 05:13:29 (7968): No heartbeat from core client for 30 sec - exiting 05:13:30 (7968): No heartbeat from core client for 30 sec - exiting 05:13:31 (7968): No heartbeat from core client for 30 sec - exiting 05:13:32 (7968): No heartbeat from core client for 30 sec - exiting 05:13:33 (7968): No heartbeat from core client for 30 sec - exiting 05:13:34 (7968): No heartbeat from core client for 30 sec - exiting 05:13:35 (7968): No heartbeat from core client for 30 sec - exiting 05:13:36 (7968): No heartbeat from core client for 30 sec - exiting 05:13:37 (7968): No heartbeat from core client for 30 sec - exiting 05:13:38 (7968): No heartbeat from core client for 30 sec - exiting 05:13:39 (7968): No heartbeat from core client for 30 sec - exiting 05:13:40 (7968): No heartbeat from core client for 30 sec - exiting 05:13:41 (7968): No heartbeat from core client for 30 sec - exiting 05:13:42 (7968): No heartbeat from core client for 30 sec - exiting 05:13:43 (7968): No heartbeat from core client for 30 sec - exiting 05:13:44 (7968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6792, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6792, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6792, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6792, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6792, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6792, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Jan 2013 02:29:32	1087577	15500392	hadcm3n_3mf1_1940_40_008264350_1	129,600	444,945	3.4332
01 Jan 2013 04:05:20	1087577	15500392	hadcm3n_3mf1_1940_40_008264350_1	103,680	356,229	3.4359
30 Dec 2012 06:43:59	1087577	15500392	hadcm3n_3mf1_1940_40_008264350_1	77,760	268,206	3.4492
28 Dec 2012 03:43:22	1087577	15500392	hadcm3n_3mf1_1940_40_008264350_1	51,840	178,925	3.4515
26 Dec 2012 02:52:41	1087577	15500392	hadcm3n_3mf1_1940_40_008264350_1	25,920	91,152	3.5167