Task 14847979

Name	hadcm3n_zj65_1880_40_008026367_1
Workunit	8181481
Created	29 Jun 2012, 16:30:41 UTC
Sent	29 Jun 2012, 16:30:49 UTC
Report deadline	28 Sep 2012, 23:58:00 UTC
Received	23 Oct 2012, 15:53:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1160812
Run time	114 days 9 hours 57 min 11 sec
CPU time	83 days 21 hours 3 min 50 sec
Validate state	Invalid
Credit	12,130.56
Device peak FLOPS	0.78 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:15:08 (3996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:00:21 (3540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:34:47 (3736): No heartbeat from core client for 30 sec - exiting 15:34:48 (3736): No heartbeat from core client for 30 sec - exiting 15:34:49 (3736): No heartbeat from core client for 30 sec - exiting 15:34:50 (3736): No heartbeat from core client for 30 sec - exiting 15:34:51 (3736): No heartbeat from core client for 30 sec - exiting 15:34:52 (3736): No heartbeat from core client for 30 sec - exiting 15:34:53 (3736): No heartbeat from core client for 30 sec - exiting 15:34:54 (3736): No heartbeat from core client for 30 sec - exiting 15:34:55 (3736): No heartbeat from core client for 30 sec - exiting 15:34:57 (3736): No heartbeat from core client for 30 sec - exiting 15:34:58 (3736): No heartbeat from core client for 30 sec - exiting 15:34:59 (3736): No heartbeat from core client for 30 sec - exiting 15:35:00 (3736): No heartbeat from core client for 30 sec - exiting 15:35:01 (3736): No heartbeat from core client for 30 sec - exiting 15:35:02 (3736): No heartbeat from core client for 30 sec - exiting 15:35:03 (3736): No heartbeat from core client for 30 sec - exiting 15:35:04 (3736): No heartbeat from core client for 30 sec - exiting 15:35:05 (3736): No heartbeat from core client for 30 sec - exiting 15:35:06 (3736): No heartbeat from core client for 30 sec - exiting 15:35:07 (3736): No heartbeat from core client for 30 sec - exiting 15:35:09 (3736): No heartbeat from core client for 30 sec - exiting 15:35:10 (3736): No heartbeat from core client for 30 sec - exiting 15:35:11 (3736): No heartbeat from core client for 30 sec - exiting 15:35:12 (3736): No heartbeat from core client for 30 sec - exiting 15:35:13 (3736): No heartbeat from core client for 30 sec - exiting 15:35:14 (3736): No heartbeat from core client for 30 sec - exiting 15:35:15 (3736): No heartbeat from core client for 30 sec - exiting 15:35:16 (3736): No heartbeat from core client for 30 sec - exiting 15:35:17 (3736): No heartbeat from core client for 30 sec - exiting 15:35:18 (3736): No heartbeat from core client for 30 sec - exiting 15:35:19 (3736): No heartbeat from core client for 30 sec - exiting 15:35:21 (3736): No heartbeat from core client for 30 sec - exiting 15:35:22 (3736): No heartbeat from core client for 30 sec - exiting 15:35:23 (3736): No heartbeat from core client for 30 sec - exiting 15:35:24 (3736): No heartbeat from core client for 30 sec - exiting 15:35:25 (3736): No heartbeat from core client for 30 sec - exiting 15:35:26 (3736): No heartbeat from core client for 30 sec - exiting 15:35:27 (3736): No heartbeat from core client for 30 sec - exiting 15:35:28 (3736): No heartbeat from core client for 30 sec - exiting 15:35:29 (3736): No heartbeat from core client for 30 sec - exiting 15:35:30 (3736): No heartbeat from core client for 30 sec - exiting 15:35:31 (3736): No heartbeat from core client for 30 sec - exiting 15:35:33 (3736): No heartbeat from core client for 30 sec - exiting 15:35:34 (3736): No heartbeat from core client for 30 sec - exiting 15:35:35 (3736): No heartbeat from core client for 30 sec - exiting 15:35:36 (3736): No heartbeat from core client for 30 sec - exiting 15:35:37 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:55:30 (2988): No heartbeat from core client for 30 sec - exiting 08:55:32 (2988): No heartbeat from core client for 30 sec - exiting 08:55:33 (2988): No heartbeat from core client for 30 sec - exiting 08:55:34 (2988): No heartbeat from core client for 30 sec - exiting 08:55:36 (2988): No heartbeat from core client for 30 sec - exiting 08:55:37 (2988): No heartbeat from core client for 30 sec - exiting 08:55:38 (2988): No heartbeat from core client for 30 sec - exiting 08:55:39 (2988): No heartbeat from core client for 30 sec - exiting 08:55:41 (2988): No heartbeat from core client for 30 sec - exiting 08:55:42 (2988): No heartbeat from core client for 30 sec - exiting 08:55:43 (2988): No heartbeat from core client for 30 sec - exiting 08:55:44 (2988): No heartbeat from core client for 30 sec - exiting 08:55:45 (2988): No heartbeat from core client for 30 sec - exiting 08:55:46 (2988): No heartbeat from core client for 30 sec - exiting 08:55:47 (2988): No heartbeat from core client for 30 sec - exiting 08:55:48 (2988): No heartbeat from core client for 30 sec - exiting 08:55:49 (2988): No heartbeat from core client for 30 sec - exiting 08:55:51 (2988): No heartbeat from core client for 30 sec - exiting 08:55:52 (2988): No heartbeat from core client for 30 sec - exiting 08:55:53 (2988): No heartbeat from core client for 30 sec - exiting 08:55:54 (2988): No heartbeat from core client for 30 sec - exiting 08:55:55 (2988): No heartbeat from core client for 30 sec - exiting 08:55:56 (2988): No heartbeat from core client for 30 sec - exiting 08:55:57 (2988): No heartbeat from core client for 30 sec - exiting 08:55:58 (2988): No heartbeat from core client for 30 sec - exiting 08:55:59 (2988): No heartbeat from core client for 30 sec - exiting 08:56:00 (2988): No heartbeat from core client for 30 sec - exiting 08:56:01 (2988): No heartbeat from core client for 30 sec - exiting 08:56:03 (2988): No heartbeat from core client for 30 sec - exiting 08:56:04 (2988): No heartbeat from core client for 30 sec - exiting 08:56:05 (2988): No heartbeat from core client for 30 sec - exiting 08:56:06 (2988): No heartbeat from core client for 30 sec - exiting 08:56:07 (2988): No heartbeat from core client for 30 sec - exiting 08:56:08 (2988): No heartbeat from core client for 30 sec - exiting 08:56:09 (2988): No heartbeat from core client for 30 sec - exiting 08:56:10 (2988): No heartbeat from core client for 30 sec - exiting 08:56:11 (2988): No heartbeat from core client for 30 sec - exiting 08:56:12 (2988): No heartbeat from core client for 30 sec - exiting 08:56:13 (2988): No heartbeat from core client for 30 sec - exiting 08:56:15 (2988): No heartbeat from core client for 30 sec - exiting 08:56:16 (2988): No heartbeat from core client for 30 sec - exiting 08:56:17 (2988): No heartbeat from core client for 30 sec - exiting 08:56:18 (2988): No heartbeat from core client for 30 sec - exiting 08:56:19 (2988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:56:56 (2124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:28:41 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:27:39 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:21:42 (3968): No heartbeat from core client for 30 sec - exiting 03:21:43 (3968): No heartbeat from core client for 30 sec - exiting 03:21:44 (3968): No heartbeat from core client for 30 sec - exiting 03:21:45 (3968): No heartbeat from core client for 30 sec - exiting 03:21:47 (3968): No heartbeat from core client for 30 sec - exiting 03:21:48 (3968): No heartbeat from core client for 30 sec - exiting 03:21:49 (3968): No heartbeat from core client for 30 sec - exiting 03:21:50 (3968): No heartbeat from core client for 30 sec - exiting 03:21:51 (3968): No heartbeat from core client for 30 sec - exiting 03:21:52 (3968): No heartbeat from core client for 30 sec - exiting 03:21:53 (3968): No heartbeat from core client for 30 sec - exiting 03:21:54 (3968): No heartbeat from core client for 30 sec - exiting 03:21:55 (3968): No heartbeat from core client for 30 sec - exiting 03:21:56 (3968): No heartbeat from core client for 30 sec - exiting 03:21:57 (3968): No heartbeat from core client for 30 sec - exiting 03:21:59 (3968): No heartbeat from core client for 30 sec - exiting 03:22:00 (3968): No heartbeat from core client for 30 sec - exiting 03:22:01 (3968): No heartbeat from core client for 30 sec - exiting 03:22:02 (3968): No heartbeat from core client for 30 sec - exiting 03:22:03 (3968): No heartbeat from core client for 30 sec - exiting 03:22:04 (3968): No heartbeat from core client for 30 sec - exiting 03:22:05 (3968): No heartbeat from core client for 30 sec - exiting 03:22:06 (3968): No heartbeat from core client for 30 sec - exiting 03:22:07 (3968): No heartbeat from core client for 30 sec - exiting 03:22:08 (3968): No heartbeat from core client for 30 sec - exiting 03:22:09 (3968): No heartbeat from core client for 30 sec - exiting 03:22:11 (3968): No heartbeat from core client for 30 sec - exiting 03:22:12 (3968): No heartbeat from core client for 30 sec - exiting 03:22:13 (3968): No heartbeat from core client for 30 sec - exiting 03:22:14 (3968): No heartbeat from core client for 30 sec - exiting 03:22:15 (3968): No heartbeat from core client for 30 sec - exiting 03:22:16 (3968): No heartbeat from core client for 30 sec - exiting 03:22:17 (3968): No heartbeat from core client for 30 sec - exiting 03:22:18 (3968): No heartbeat from core client for 30 sec - exiting 03:22:19 (3968): No heartbeat from core client for 30 sec - exiting 03:22:20 (3968): No heartbeat from core client for 30 sec - exiting 03:22:21 (3968): No heartbeat from core client for 30 sec - exiting 03:22:23 (3968): No heartbeat from core client for 30 sec - exiting 03:22:24 (3968): No heartbeat from core client for 30 sec - exiting 03:22:25 (3968): No heartbeat from core client for 30 sec - exiting 03:22:26 (3968): No heartbeat from core client for 30 sec - exiting 03:22:27 (3968): No heartbeat from core client for 30 sec - exiting 03:22:28 (3968): No heartbeat from core client for 30 sec - exiting 03:22:29 (3968): No heartbeat from core client for 30 sec - exiting 03:22:30 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3208, iMonCtr=1 Model crash detected, will try to restart... 07:40:34 (3956): No heartbeat from core client for 30 sec - exiting 07:40:36 (3956): No heartbeat from core client for 30 sec - exiting 07:40:37 (3956): No heartbeat from core client for 30 sec - exiting 07:40:38 (3956): No heartbeat from core client for 30 sec - exiting 07:40:39 (3956): No heartbeat from core client for 30 sec - exiting 07:40:40 (3956): No heartbeat from core client for 30 sec - exiting 07:40:41 (3956): No heartbeat from core client for 30 sec - exiting 07:40:42 (3956): No heartbeat from core client for 30 sec - exiting 07:40:43 (3956): No heartbeat from core client for 30 sec - exiting 07:40:44 (3956): No heartbeat from core client for 30 sec - exiting 07:40:45 (3956): No heartbeat from core client for 30 sec - exiting 07:40:47 (3956): No heartbeat from core client for 30 sec - exiting 07:40:48 (3956): No heartbeat from core client for 30 sec - exiting 07:40:49 (3956): No heartbeat from core client for 30 sec - exiting 07:40:50 (3956): No heartbeat from core client for 30 sec - exiting 07:40:51 (3956): No heartbeat from core client for 30 sec - exiting 07:40:52 (3956): No heartbeat from core client for 30 sec - exiting 07:40:53 (3956): No heartbeat from core client for 30 sec - exiting 07:40:54 (3956): No heartbeat from core client for 30 sec - exiting 07:40:55 (3956): No heartbeat from core client for 30 sec - exiting 07:40:56 (3956): No heartbeat from core client for 30 sec - exiting 07:40:57 (3956): No heartbeat from core client for 30 sec - exiting 07:40:59 (3956): No heartbeat from core client for 30 sec - exiting 07:41:00 (3956): No heartbeat from core client for 30 sec - exiting 07:41:01 (3956): No heartbeat from core client for 30 sec - exiting 07:41:02 (3956): No heartbeat from core client for 30 sec - exiting 07:41:03 (3956): No heartbeat from core client for 30 sec - exiting 07:41:04 (3956): No heartbeat from core client for 30 sec - exiting 07:41:05 (3956): No heartbeat from core client for 30 sec - exiting 07:41:06 (3956): No heartbeat from core client for 30 sec - exiting 07:41:07 (3956): No heartbeat from core client for 30 sec - exiting 07:41:08 (3956): No heartbeat from core client for 30 sec - exiting 07:41:09 (3956): No heartbeat from core client for 30 sec - exiting 07:41:11 (3956): No heartbeat from core client for 30 sec - exiting 07:41:12 (3956): No heartbeat from core client for 30 sec - exiting 07:41:13 (3956): No heartbeat from core client for 30 sec - exiting 07:41:14 (3956): No heartbeat from core client for 30 sec - exiting 07:41:15 (3956): No heartbeat from core client for 30 sec - exiting 07:41:16 (3956): No heartbeat from core client for 30 sec - exiting 07:41:17 (3956): No heartbeat from core client for 30 sec - exiting 07:41:18 (3956): No heartbeat from core client for 30 sec - exiting 07:41:19 (3956): No heartbeat from core client for 30 sec - exiting 07:41:20 (3956): No heartbeat from core client for 30 sec - exiting 07:41:21 (3956): No heartbeat from core client for 30 sec - exiting 07:41:23 (3956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:08:16 (660): No heartbeat from core client for 30 sec - exiting 10:08:17 (660): No heartbeat from core client for 30 sec - exiting 10:08:18 (660): No heartbeat from core client for 30 sec - exiting 10:08:19 (660): No heartbeat from core client for 30 sec - exiting 10:08:20 (660): No heartbeat from core client for 30 sec - exiting 10:08:21 (660): No heartbeat from core client for 30 sec - exiting 10:08:23 (660): No heartbeat from core client for 30 sec - exiting 10:08:24 (660): No heartbeat from core client for 30 sec - exiting 10:08:25 (660): No heartbeat from core client for 30 sec - exiting 10:08:26 (660): No heartbeat from core client for 30 sec - exiting 10:08:27 (660): No heartbeat from core client for 30 sec - exiting 10:08:28 (660): No heartbeat from core client for 30 sec - exiting 10:08:29 (660): No heartbeat from core client for 30 sec - exiting 10:08:30 (660): No heartbeat from core client for 30 sec - exiting 10:08:31 (660): No heartbeat from core client for 30 sec - exiting 10:08:32 (660): No heartbeat from core client for 30 sec - exiting 10:08:33 (660): No heartbeat from core client for 30 sec - exiting 10:08:35 (660): No heartbeat from core client for 30 sec - exiting 10:08:36 (660): No heartbeat from core client for 30 sec - exiting 10:08:37 (660): No heartbeat from core client for 30 sec - exiting 10:08:38 (660): No heartbeat from core client for 30 sec - exiting 10:08:39 (660): No heartbeat from core client for 30 sec - exiting 10:08:40 (660): No heartbeat from core client for 30 sec - exiting 10:08:41 (660): No heartbeat from core client for 30 sec - exiting 10:08:42 (660): No heartbeat from core client for 30 sec - exiting 10:08:43 (660): No heartbeat from core client for 30 sec - exiting 10:08:44 (660): No heartbeat from core client for 30 sec - exiting 10:08:45 (660): No heartbeat from core client for 30 sec - exiting 10:08:47 (660): No heartbeat from core client for 30 sec - exiting 10:08:48 (660): No heartbeat from core client for 30 sec - exiting 10:08:49 (660): No heartbeat from core client for 30 sec - exiting 10:08:50 (660): No heartbeat from core client for 30 sec - exiting 10:08:51 (660): No heartbeat from core client for 30 sec - exiting 10:08:52 (660): No heartbeat from core client for 30 sec - exiting 10:08:53 (660): No heartbeat from core client for 30 sec - exiting 10:08:54 (660): No heartbeat from core client for 30 sec - exiting 10:08:55 (660): No heartbeat from core client for 30 sec - exiting 10:08:56 (660): No heartbeat from core client for 30 sec - exiting 10:08:58 (660): No heartbeat from core client for 30 sec - exiting 10:08:59 (660): No heartbeat from core client for 30 sec - exiting 10:09:00 (660): No heartbeat from core client for 30 sec - exiting 10:09:01 (660): No heartbeat from core client for 30 sec - exiting 10:09:02 (660): No heartbeat from core client for 30 sec - exiting 10:09:03 (660): No heartbeat from core client for 30 sec - exiting 10:09:04 (660): No heartbeat from core client for 30 sec - exiting 10:09:05 (660): No heartbeat from core client for 30 sec - exiting 10:09:06 (660): No heartbeat from core client for 30 sec - exiting 10:09:07 (660): No heartbeat from core client for 30 sec - exiting 10:09:08 (660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:04:02 (3120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:03:39 (5896): No heartbeat from core client for 30 sec - exiting 12:03:40 (5896): No heartbeat from core client for 30 sec - exiting 12:03:41 (5896): No heartbeat from core client for 30 sec - exiting 12:03:42 (5896): No heartbeat from core client for 30 sec - exiting 12:03:43 (5896): No heartbeat from core client for 30 sec - exiting 12:03:44 (5896): No heartbeat from core client for 30 sec - exiting 12:03:45 (5896): No heartbeat from core client for 30 sec - exiting 12:03:46 (5896): No heartbeat from core client for 30 sec - exiting 12:03:47 (5896): No heartbeat from core client for 30 sec - exiting 12:03:48 (5896): No heartbeat from core client for 30 sec - exiting 12:03:49 (5896): No heartbeat from core client for 30 sec - exiting 12:03:50 (5896): No heartbeat from core client for 30 sec - exiting 12:03:51 (5896): No heartbeat from core client for 30 sec - exiting 12:03:52 (5896): No heartbeat from core client for 30 sec - exiting 12:03:53 (5896): No heartbeat from core client for 30 sec - exiting 12:03:54 (5896): No heartbeat from core client for 30 sec - exiting 12:03:55 (5896): No heartbeat from core client for 30 sec - exiting 12:03:56 (5896): No heartbeat from core client for 30 sec - exiting 12:03:57 (5896): No heartbeat from core client for 30 sec - exiting 12:03:58 (5896): No heartbeat from core client for 30 sec - exiting 12:03:59 (5896): No heartbeat from core client for 30 sec - exiting 12:04:00 (5896): No heartbeat from core client for 30 sec - exiting 12:04:01 (5896): No heartbeat from core client for 30 sec - exiting 12:04:02 (5896): No heartbeat from core client for 30 sec - exiting 12:04:03 (5896): No heartbeat from core client for 30 sec - exiting 12:04:04 (5896): No heartbeat from core client for 30 sec - exiting 12:04:05 (5896): No heartbeat from core client for 30 sec - exiting 12:04:06 (5896): No heartbeat from core client for 30 sec - exiting 12:04:07 (5896): No heartbeat from core client for 30 sec - exiting 12:04:08 (5896): No heartbeat from core client for 30 sec - exiting 12:04:09 (5896): No heartbeat from core client for 30 sec - exiting 12:04:10 (5896): No heartbeat from core client for 30 sec - exiting 12:04:11 (5896): No heartbeat from core client for 30 sec - exiting 12:04:12 (5896): No heartbeat from core client for 30 sec - exiting 12:04:13 (5896): No heartbeat from core client for 30 sec - exiting 12:04:14 (5896): No heartbeat from core client for 30 sec - exiting 12:04:15 (5896): No heartbeat from core client for 30 sec - exiting 12:04:16 (5896): No heartbeat from core client for 30 sec - exiting 12:04:17 (5896): No heartbeat from core client for 30 sec - exiting 12:04:18 (5896): No heartbeat from core client for 30 sec - exiting 12:04:19 (5896): No heartbeat from core client for 30 sec - exiting 12:04:20 (5896): No heartbeat from core client for 30 sec - exiting 12:04:21 (5896): No heartbeat from core client for 30 sec - exiting 12:04:22 (5896): No heartbeat from core client for 30 sec - exiting 12:04:23 (5896): No heartbeat from core client for 30 sec - exiting 12:04:24 (5896): No heartbeat from core client for 30 sec - exiting 12:04:25 (5896): No heartbeat from core client for 30 sec - exiting 12:04:26 (5896): No heartbeat from core client for 30 sec - exiting 12:04:27 (5896): No heartbeat from core client for 30 sec - exiting 12:04:28 (5896): No heartbeat from core client for 30 sec - exiting 12:04:29 (5896): No heartbeat from core client for 30 sec - exiting 12:04:30 (5896): No heartbeat from core client for 30 sec - exiting 12:04:31 (5896): No heartbeat from core client for 30 sec - exiting 12:04:32 (5896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:26:58 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:06:31 (5248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5628, iMonCtr=1 Model crash detected, will try to restart... 11:21:55 (5996): No heartbeat from core client for 30 sec - exiting 11:21:56 (5996): No heartbeat from core client for 30 sec - exiting 11:21:57 (5996): No heartbeat from core client for 30 sec - exiting 11:21:59 (5996): No heartbeat from core client for 30 sec - exiting 11:22:00 (5996): No heartbeat from core client for 30 sec - exiting 11:22:01 (5996): No heartbeat from core client for 30 sec - exiting 11:22:02 (5996): No heartbeat from core client for 30 sec - exiting 11:22:04 (5996): No heartbeat from core client for 30 sec - exiting 11:22:05 (5996): No heartbeat from core client for 30 sec - exiting 11:22:06 (5996): No heartbeat from core client for 30 sec - exiting 11:22:07 (5996): No heartbeat from core client for 30 sec - exiting 11:22:09 (5996): No heartbeat from core client for 30 sec - exiting 11:22:10 (5996): No heartbeat from core client for 30 sec - exiting 11:22:11 (5996): No heartbeat from core client for 30 sec - exiting 11:22:12 (5996): No heartbeat from core client for 30 sec - exiting 11:22:14 (5996): No heartbeat from core client for 30 sec - exiting 11:22:15 (5996): No heartbeat from core client for 30 sec - exiting 11:22:16 (5996): No heartbeat from core client for 30 sec - exiting 11:22:17 (5996): No heartbeat from core client for 30 sec - exiting 11:22:18 (5996): No heartbeat from core client for 30 sec - exiting 11:22:19 (5996): No heartbeat from core client for 30 sec - exiting 11:22:20 (5996): No heartbeat from core client for 30 sec - exiting 11:22:21 (5996): No heartbeat from core client for 30 sec - exiting 11:22:22 (5996): No heartbeat from core client for 30 sec - exiting 11:22:23 (5996): No heartbeat from core client for 30 sec - exiting 11:22:24 (5996): No heartbeat from core client for 30 sec - exiting 11:22:26 (5996): No heartbeat from core client for 30 sec - exiting 11:22:27 (5996): No heartbeat from core client for 30 sec - exiting 11:22:28 (5996): No heartbeat from core client for 30 sec - exiting 11:22:29 (5996): No heartbeat from core client for 30 sec - exiting 11:22:30 (5996): No heartbeat from core client for 30 sec - exiting 11:22:31 (5996): No heartbeat from core client for 30 sec - exiting 11:22:32 (5996): No heartbeat from core client for 30 sec - exiting 11:22:33 (5996): No heartbeat from core client for 30 sec - exiting 11:22:34 (5996): No heartbeat from core client for 30 sec - exiting 11:22:35 (5996): No heartbeat from core client for 30 sec - exiting 11:22:37 (5996): No heartbeat from core client for 30 sec - exiting 11:22:38 (5996): No heartbeat from core client for 30 sec - exiting 11:22:39 (5996): No heartbeat from core client for 30 sec - exiting 11:22:40 (5996): No heartbeat from core client for 30 sec - exiting 11:22:41 (5996): No heartbeat from core client for 30 sec - exiting 11:22:42 (5996): No heartbeat from core client for 30 sec - exiting 11:22:43 (5996): No heartbeat from core client for 30 sec - exiting 11:22:44 (5996): No heartbeat from core client for 30 sec - exiting 11:22:45 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6120, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 00:13:36 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:13:28 (5588): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:52:35 (1032): No heartbeat from core client for 30 sec - exiting 10:52:36 (1032): No heartbeat from core client for 30 sec - exiting 10:52:37 (1032): No heartbeat from core client for 30 sec - exiting 10:52:38 (1032): No heartbeat from core client for 30 sec - exiting 10:52:39 (1032): No heartbeat from core client for 30 sec - exiting 10:52:40 (1032): No heartbeat from core client for 30 sec - exiting 10:52:41 (1032): No heartbeat from core client for 30 sec - exiting 10:52:42 (1032): No heartbeat from core client for 30 sec - exiting 10:52:43 (1032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3776, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy 2048 Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy 2048 Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy 2048 Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy 2048 Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy 2048 Model crashed: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Oct 2012 10:38:25	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	1,010,880	8,026,008	7.9396
20 Oct 2012 20:02:29	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	984,960	7,823,520	7.9430
18 Oct 2012 06:16:38	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	959,040	7,620,865	7.9463
15 Oct 2012 13:06:16	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	933,120	7,415,666	7.9472
12 Oct 2012 08:05:17	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	907,200	7,205,498	7.9426
08 Oct 2012 19:39:06	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	881,280	6,990,685	7.9324
05 Oct 2012 01:20:56	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	855,360	6,775,519	7.9212
01 Oct 2012 23:09:29	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	829,440	6,565,387	7.9154
28 Sep 2012 21:22:59	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	803,520	6,356,405	7.9107
24 Sep 2012 19:22:47	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	777,600	6,147,200	7.9053
22 Sep 2012 00:56:13	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	751,680	5,942,660	7.9058
19 Sep 2012 10:13:28	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	725,760	5,737,473	7.9055
17 Sep 2012 00:08:10	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	699,840	5,536,343	7.9109
14 Sep 2012 11:13:01	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	673,920	5,333,584	7.9143
11 Sep 2012 23:47:27	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	648,000	5,131,790	7.9194
09 Sep 2012 08:14:58	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	622,080	4,928,671	7.9229
06 Sep 2012 19:29:10	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	596,160	4,728,286	7.9312
04 Sep 2012 09:06:11	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	570,240	4,529,411	7.9430
01 Sep 2012 21:32:53	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	544,320	4,328,865	7.9528
30 Aug 2012 11:21:38	1160812	14847979	hadcm3n_zj65_1880_40_008026367_1	518,400	4,130,668	7.9681