Task 13631485

Name	hadcm3n_o6vw_1980_40_007545056_1
Workunit	7742288
Created	10 Nov 2011, 7:58:19 UTC
Sent	10 Nov 2011, 8:03:02 UTC
Report deadline	9 Feb 2012, 15:30:13 UTC
Received	18 Dec 2011, 16:34:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1179386
Run time	12 days 15 hours 37 min 51 sec
CPU time	12 days 13 hours 59 min 49 sec
Validate state	Invalid
Credit	11,197.44
Device peak FLOPS	2.47 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:08:15 (9108): No heartbeat from core client for 30 sec - exiting 21:08:16 (9108): No heartbeat from core client for 30 sec - exiting 21:08:17 (9108): No heartbeat from core client for 30 sec - exiting 21:08:19 (9108): No heartbeat from core client for 30 sec - exiting 21:08:20 (9108): No heartbeat from core client for 30 sec - exiting 21:08:21 (9108): No heartbeat from core client for 30 sec - exiting 21:08:22 (9108): No heartbeat from core client for 30 sec - exiting 21:08:23 (9108): No heartbeat from core client for 30 sec - exiting 21:08:24 (9108): No heartbeat from core client for 30 sec - exiting 21:08:25 (9108): No heartbeat from core client for 30 sec - exiting 21:08:26 (9108): No heartbeat from core client for 30 sec - exiting 21:08:27 (9108): No heartbeat from core client for 30 sec - exiting 21:08:28 (9108): No heartbeat from core client for 30 sec - exiting 21:08:29 (9108): No heartbeat from core client for 30 sec - exiting 21:08:31 (9108): No heartbeat from core client for 30 sec - exiting 21:08:32 (9108): No heartbeat from core client for 30 sec - exiting 21:08:33 (9108): No heartbeat from core client for 30 sec - exiting 21:08:34 (9108): No heartbeat from core client for 30 sec - exiting 21:08:35 (9108): No heartbeat from core client for 30 sec - exiting 21:08:36 (9108): No heartbeat from core client for 30 sec - exiting 21:08:37 (9108): No heartbeat from core client for 30 sec - exiting 21:08:38 (9108): No heartbeat from core client for 30 sec - exiting 21:08:39 (9108): No heartbeat from core client for 30 sec - exiting 21:08:40 (9108): No heartbeat from core client for 30 sec - exiting 21:08:41 (9108): No heartbeat from core client for 30 sec - exiting 21:08:42 (9108): No heartbeat from core client for 30 sec - exiting 21:08:44 (9108): No heartbeat from core client for 30 sec - exiting 21:08:45 (9108): No heartbeat from core client for 30 sec - exiting 21:08:46 (9108): No heartbeat from core client for 30 sec - exiting 21:08:47 (9108): No heartbeat from core client for 30 sec - exiting 21:08:48 (9108): No heartbeat from core client for 30 sec - exiting 21:08:49 (9108): No heartbeat from core client for 30 sec - exiting 21:08:50 (9108): No heartbeat from core client for 30 sec - exiting 21:08:51 (9108): No heartbeat from core client for 30 sec - exiting 21:08:52 (9108): No heartbeat from core client for 30 sec - exiting 21:08:53 (9108): No heartbeat from core client for 30 sec - exiting 21:08:54 (9108): No heartbeat from core client for 30 sec - exiting 21:08:55 (9108): No heartbeat from core client for 30 sec - exiting 21:08:56 (9108): No heartbeat from core client for 30 sec - exiting 21:08:57 (9108): No heartbeat from core client for 30 sec - exiting 21:08:59 (9108): No heartbeat from core client for 30 sec - exiting 21:09:00 (9108): No heartbeat from core client for 30 sec - exiting 21:09:01 (9108): No heartbeat from core client for 30 sec - exiting 21:09:02 (9108): No heartbeat from core client for 30 sec - exiting 21:09:03 (9108): No heartbeat from core client for 30 sec - exiting 21:09:04 (9108): No heartbeat from core client for 30 sec - exiting 21:09:05 (9108): No heartbeat from core client for 30 sec - exiting 21:09:06 (9108): No heartbeat from core client for 30 sec - exiting 21:09:07 (9108): No heartbeat from core client for 30 sec - exiting 21:09:08 (9108): No heartbeat from core client for 30 sec - exiting 21:09:09 (9108): No heartbeat from core client for 30 sec - exiting 21:09:11 (9108): No heartbeat from core client for 30 sec - exiting 21:09:12 (9108): No heartbeat from core client for 30 sec - exiting 21:09:13 (9108): No heartbeat from core client for 30 sec - exiting 21:09:14 (9108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:03:02 (5944): No heartbeat from core client for 30 sec - exiting 19:03:03 (5944): No heartbeat from core client for 30 sec - exiting 19:03:04 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:01:33 (6568): No heartbeat from core client for 30 sec - exiting 19:01:34 (6568): No heartbeat from core client for 30 sec - exiting 19:01:35 (6568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:32 (6800): No heartbeat from core client for 30 sec - exiting 20:14:33 (6800): No heartbeat from core client for 30 sec - exiting 20:14:34 (6800): No heartbeat from core client for 30 sec - exiting 20:14:35 (6800): No heartbeat from core client for 30 sec - exiting 20:14:36 (6800): No heartbeat from core client for 30 sec - exiting 20:14:38 (6800): No heartbeat from core client for 30 sec - exiting 20:14:39 (6800): No heartbeat from core client for 30 sec - exiting 20:14:40 (6800): No heartbeat from core client for 30 sec - exiting 20:14:41 (6800): No heartbeat from core client for 30 sec - exiting 20:14:42 (6800): No heartbeat from core client for 30 sec - exiting 20:14:43 (6800): No heartbeat from core client for 30 sec - exiting 20:14:44 (6800): No heartbeat from core client for 30 sec - exiting 20:14:45 (6800): No heartbeat from core client for 30 sec - exiting 20:14:46 (6800): No heartbeat from core client for 30 sec - exiting 20:14:47 (6800): No heartbeat from core client for 30 sec - exiting 20:14:48 (6800): No heartbeat from core client for 30 sec - exiting 20:14:50 (6800): No heartbeat from core client for 30 sec - exiting 20:14:51 (6800): No heartbeat from core client for 30 sec - exiting 20:14:52 (6800): No heartbeat from core client for 30 sec - exiting 20:14:53 (6800): No heartbeat from core client for 30 sec - exiting 20:14:54 (6800): No heartbeat from core client for 30 sec - exiting 20:14:55 (6800): No heartbeat from core client for 30 sec - exiting 20:14:56 (6800): No heartbeat from core client for 30 sec - exiting 20:14:57 (6800): No heartbeat from core client for 30 sec - exiting 20:14:58 (6800): No heartbeat from core client for 30 sec - exiting 20:14:59 (6800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:15:00 (7524): No heartbeat from core client for 30 sec - exiting 21:15:01 (7524): No heartbeat from core client for 30 sec - exiting 21:15:02 (7524): No heartbeat from core client for 30 sec - exiting 21:15:03 (7524): No heartbeat from core client for 30 sec - exiting 21:15:04 (7524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:15:05 (7524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:04:35 (7960): No heartbeat from core client for 30 sec - exiting 11:04:36 (7960): No heartbeat from core client for 30 sec - exiting 11:04:37 (7960): No heartbeat from core client for 30 sec - exiting 11:04:38 (7960): No heartbeat from core client for 30 sec - exiting 11:04:39 (7960): No heartbeat from core client for 30 sec - exiting 11:04:40 (7960): No heartbeat from core client for 30 sec - exiting 11:04:41 (7960): No heartbeat from core client for 30 sec - exiting 11:04:42 (7960): No heartbeat from core client for 30 sec - exiting 11:04:43 (7960): No heartbeat from core client for 30 sec - exiting 11:04:44 (7960): No heartbeat from core client for 30 sec - exiting 11:04:45 (7960): No heartbeat from core client for 30 sec - exiting 11:04:46 (7960): No heartbeat from core client for 30 sec - exiting 11:04:47 (7960): No heartbeat from core client for 30 sec - exiting 11:04:48 (7960): No heartbeat from core client for 30 sec - exiting 11:04:49 (7960): No heartbeat from core client for 30 sec - exiting 11:04:50 (7960): No heartbeat from core client for 30 sec - exiting 11:04:51 (7960): No heartbeat from core client for 30 sec - exiting 11:04:52 (7960): No heartbeat from core client for 30 sec - exiting 11:04:53 (7960): No heartbeat from core client for 30 sec - exiting 11:04:54 (7960): No heartbeat from core client for 30 sec - exiting 11:04:55 (7960): No heartbeat from core client for 30 sec - exiting 11:04:56 (7960): No heartbeat from core client for 30 sec - exiting 11:04:57 (7960): No heartbeat from core client for 30 sec - exiting 11:04:58 (7960): No heartbeat from core client for 30 sec - exiting 11:04:59 (7960): No heartbeat from core client for 30 sec - exiting 11:05:00 (7960): No heartbeat from core client for 30 sec - exiting 11:05:01 (7960): No heartbeat from core client for 30 sec - exiting 11:05:02 (7960): No heartbeat from core client for 30 sec - exiting 11:05:03 (7960): No heartbeat from core client for 30 sec - exiting 11:05:04 (7960): No heartbeat from core client for 30 sec - exiting 11:05:05 (7960): No heartbeat from core client for 30 sec - exiting 11:05:07 (7960): No heartbeat from core client for 30 sec - exiting 11:05:08 (7960): No heartbeat from core client for 30 sec - exiting 11:05:09 (7960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:42:21 (7940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:46:46 (6048): No heartbeat from core client for 30 sec - exiting 09:46:47 (6048): No heartbeat from core client for 30 sec - exiting 09:46:48 (6048): No heartbeat from core client for 30 sec - exiting 09:46:49 (6048): No heartbeat from core client for 30 sec - exiting 09:46:50 (6048): No heartbeat from core client for 30 sec - exiting 09:46:51 (6048): No heartbeat from core client for 30 sec - exiting 09:46:52 (6048): No heartbeat from core client for 30 sec - exiting 09:46:53 (6048): No heartbeat from core client for 30 sec - exiting 09:46:54 (6048): No heartbeat from core client for 30 sec - exiting 09:46:55 (6048): No heartbeat from core client for 30 sec - exiting 09:46:56 (6048): No heartbeat from core client for 30 sec - exiting 09:46:57 (6048): No heartbeat from core client for 30 sec - exiting 09:46:58 (6048): No heartbeat from core client for 30 sec - exiting 09:46:59 (6048): No heartbeat from core client for 30 sec - exiting 09:47:00 (6048): No heartbeat from core client for 30 sec - exiting 09:47:01 (6048): No heartbeat from core client for 30 sec - exiting 09:47:02 (6048): No heartbeat from core client for 30 sec - exiting 09:47:03 (6048): No heartbeat from core client for 30 sec - exiting 09:47:04 (6048): No heartbeat from core client for 30 sec - exiting 09:47:05 (6048): No heartbeat from core client for 30 sec - exiting 09:47:06 (6048): No heartbeat from core client for 30 sec - exiting 09:47:07 (6048): No heartbeat from core client for 30 sec - exiting 09:47:08 (6048): No heartbeat from core client for 30 sec - exiting 09:47:09 (6048): No heartbeat from core client for 30 sec - exiting 09:47:10 (6048): No heartbeat from core client for 30 sec - exiting 09:47:11 (6048): No heartbeat from core client for 30 sec - exiting 09:47:12 (6048): No heartbeat from core client for 30 sec - exiting 09:47:13 (6048): No heartbeat from core client for 30 sec - exiting 09:47:14 (6048): No heartbeat from core client for 30 sec - exiting 09:47:15 (6048): No heartbeat from core client for 30 sec - exiting 09:47:16 (6048): No heartbeat from core client for 30 sec - exiting 09:47:17 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:41:02 (8032): No heartbeat from core client for 30 sec - exiting 15:41:03 (8032): No heartbeat from core client for 30 sec - exiting 15:41:04 (8032): No heartbeat from core client for 30 sec - exiting 15:41:05 (8032): No heartbeat from core client for 30 sec - exiting 15:41:06 (8032): No heartbeat from core client for 30 sec - exiting 15:41:07 (8032): No heartbeat from core client for 30 sec - exiting 15:41:08 (8032): No heartbeat from core client for 30 sec - exiting 15:41:09 (8032): No heartbeat from core client for 30 sec - exiting 15:41:10 (8032): No heartbeat from core client for 30 sec - exiting 15:41:11 (8032): No heartbeat from core client for 30 sec - exiting 15:41:12 (8032): No heartbeat from core client for 30 sec - exiting 15:41:13 (8032): No heartbeat from core client for 30 sec - exiting 15:41:14 (8032): No heartbeat from core client for 30 sec - exiting 15:41:15 (8032): No heartbeat from core client for 30 sec - exiting 15:41:16 (8032): No heartbeat from core client for 30 sec - exiting 15:41:17 (8032): No heartbeat from core client for 30 sec - exiting 15:41:18 (8032): No heartbeat from core client for 30 sec - exiting 15:41:19 (8032): No heartbeat from core client for 30 sec - exiting 15:41:20 (8032): No heartbeat from core client for 30 sec - exiting 15:41:21 (8032): No heartbeat from core client for 30 sec - exiting 15:41:22 (8032): No heartbeat from core client for 30 sec - exiting 15:41:23 (8032): No heartbeat from core client for 30 sec - exiting 15:41:24 (8032): No heartbeat from core client for 30 sec - exiting 15:41:25 (8032): No heartbeat from core client for 30 sec - exiting 15:41:26 (8032): No heartbeat from core client for 30 sec - exiting 15:41:27 (8032): No heartbeat from core client for 30 sec - exiting 15:41:28 (8032): No heartbeat from core client for 30 sec - exiting 15:41:29 (8032): No heartbeat from core client for 30 sec - exiting 15:41:30 (8032): No heartbeat from core client for 30 sec - exiting 15:41:31 (8032): No heartbeat from core client for 30 sec - exiting 15:41:32 (8032): No heartbeat from core client for 30 sec - exiting 15:41:33 (8032): No heartbeat from core client for 30 sec - exiting 15:41:34 (8032): No heartbeat from core client for 30 sec - exiting 15:41:35 (8032): No heartbeat from core client for 30 sec - exiting 15:41:36 (8032): No heartbeat from core client for 30 sec - exiting 15:41:37 (8032): No heartbeat from core client for 30 sec - exiting 15:41:38 (8032): No heartbeat from core client for 30 sec - exiting 15:41:39 (8032): No heartbeat from core client for 30 sec - exiting 15:41:40 (8032): No heartbeat from core client for 30 sec - exiting 15:41:41 (8032): No heartbeat from core client for 30 sec - exiting 15:41:42 (8032): No heartbeat from core client for 30 sec - exiting 15:41:43 (8032): No heartbeat from core client for 30 sec - exiting 15:41:44 (8032): No heartbeat from core client for 30 sec - exiting 15:41:45 (8032): No heartbeat from core client for 30 sec - exiting 15:41:46 (8032): No heartbeat from core client for 30 sec - exiting 15:41:47 (8032): No heartbeat from core client for 30 sec - exiting 15:41:48 (8032): No heartbeat from core client for 30 sec - exiting 15:41:49 (8032): No heartbeat from core client for 30 sec - exiting 15:41:50 (8032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:33:24 (6916): No heartbeat from core client for 30 sec - exiting 12:33:26 (6916): No heartbeat from core client for 30 sec - exiting 12:33:27 (6916): No heartbeat from core client for 30 sec - exiting 12:33:28 (6916): No heartbeat from core client for 30 sec - exiting 12:33:29 (6916): No heartbeat from core client for 30 sec - exiting 12:33:30 (6916): No heartbeat from core client for 30 sec - exiting 12:33:31 (6916): No heartbeat from core client for 30 sec - exiting 12:33:32 (6916): No heartbeat from core client for 30 sec - exiting 12:33:34 (6916): No heartbeat from core client for 30 sec - exiting 12:33:35 (6916): No heartbeat from core client for 30 sec - exiting 12:33:36 (6916): No heartbeat from core client for 30 sec - exiting 12:33:37 (6916): No heartbeat from core client for 30 sec - exiting 12:33:38 (6916): No heartbeat from core client for 30 sec - exiting 12:33:39 (6916): No heartbeat from core client for 30 sec - exiting 12:33:40 (6916): No heartbeat from core client for 30 sec - exiting 12:33:41 (6916): No heartbeat from core client for 30 sec - exiting 12:33:42 (6916): No heartbeat from core client for 30 sec - exiting 12:33:43 (6916): No heartbeat from core client for 30 sec - exiting 12:33:44 (6916): No heartbeat from core client for 30 sec - exiting 12:33:45 (6916): No heartbeat from core client for 30 sec - exiting 12:33:46 (6916): No heartbeat from core client for 30 sec - exiting 12:33:47 (6916): No heartbeat from core client for 30 sec - exiting 12:33:48 (6916): No heartbeat from core client for 30 sec - exiting 12:33:49 (6916): No heartbeat from core client for 30 sec - exiting 12:33:50 (6916): No heartbeat from core client for 30 sec - exiting 12:33:51 (6916): No heartbeat from core client for 30 sec - exiting 12:33:52 (6916): No heartbeat from core client for 30 sec - exiting 12:33:53 (6916): No heartbeat from core client for 30 sec - exiting 12:33:54 (6916): No heartbeat from core client for 30 sec - exiting 12:33:55 (6916): No heartbeat from core client for 30 sec - exiting 12:33:56 (6916): No heartbeat from core client for 30 sec - exiting 12:33:57 (6916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:55:09 (2976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:10:19 (4740): No heartbeat from core client for 30 sec - exiting 17:10:20 (4740): No heartbeat from core client for 30 sec - exiting 17:10:21 (4740): No heartbeat from core client for 30 sec - exiting 17:10:22 (4740): No heartbeat from core client for 30 sec - exiting 17:10:24 (4740): No heartbeat from core client for 30 sec - exiting 17:10:25 (4740): No heartbeat from core client for 30 sec - exiting 17:10:26 (4740): No heartbeat from core client for 30 sec - exiting 17:10:27 (4740): No heartbeat from core client for 30 sec - exiting 17:10:28 (4740): No heartbeat from core client for 30 sec - exiting 17:10:29 (4740): No heartbeat from core client for 30 sec - exiting 17:10:30 (4740): No heartbeat from core client for 30 sec - exiting 17:10:31 (4740): No heartbeat from core client for 30 sec - exiting 17:10:32 (4740): No heartbeat from core client for 30 sec - exiting 17:10:33 (4740): No heartbeat from core client for 30 sec - exiting 17:10:34 (4740): No heartbeat from core client for 30 sec - exiting 17:10:36 (4740): No heartbeat from core client for 30 sec - exiting 17:10:37 (4740): No heartbeat from core client for 30 sec - exiting 17:10:38 (4740): No heartbeat from core client for 30 sec - exiting 17:10:39 (4740): No heartbeat from core client for 30 sec - exiting 17:10:40 (4740): No heartbeat from core client for 30 sec - exiting 17:10:41 (4740): No heartbeat from core client for 30 sec - exiting 17:10:42 (4740): No heartbeat from core client for 30 sec - exiting 17:10:43 (4740): No heartbeat from core client for 30 sec - exiting 17:10:44 (4740): No heartbeat from core client for 30 sec - exiting 17:10:45 (4740): No heartbeat from core client for 30 sec - exiting 17:10:46 (4740): No heartbeat from core client for 30 sec - exiting 17:10:48 (4740): No heartbeat from core client for 30 sec - exiting 17:10:49 (4740): No heartbeat from core client for 30 sec - exiting 17:10:50 (4740): No heartbeat from core client for 30 sec - exiting 17:10:51 (4740): No heartbeat from core client for 30 sec - exiting 17:10:52 (4740): No heartbeat from core client for 30 sec - exiting 17:10:53 (4740): No heartbeat from core client for 30 sec - exiting 17:10:54 (4740): No heartbeat from core client for 30 sec - exiting 17:10:55 (4740): No heartbeat from core client for 30 sec - exiting 17:10:56 (4740): No heartbeat from core client for 30 sec - exiting 17:10:57 (4740): No heartbeat from core client for 30 sec - exiting 17:10:58 (4740): No heartbeat from core client for 30 sec - exiting 17:11:00 (4740): No heartbeat from core client for 30 sec - exiting 17:11:01 (4740): No heartbeat from core client for 30 sec - exiting 17:11:02 (4740): No heartbeat from core client for 30 sec - exiting 17:11:03 (4740): No heartbeat from core client for 30 sec - exiting 17:11:04 (4740): No heartbeat from core client for 30 sec - exiting 17:11:05 (4740): No heartbeat from core client for 30 sec - exiting 17:11:06 (4740): No heartbeat from core client for 30 sec - exiting 17:11:07 (4740): No heartbeat from core client for 30 sec - exiting 17:11:08 (4740): No heartbeat from core client for 30 sec - exiting 17:11:09 (4740): No heartbeat from core client for 30 sec - exiting 17:11:10 (4740): No heartbeat from core client for 30 sec - exiting 17:11:12 (4740): No heartbeat from core client for 30 sec - exiting 17:11:13 (4740): No heartbeat from core client for 30 sec - exiting 17:11:14 (4740): No heartbeat from core client for 30 sec - exiting 17:11:15 (4740): No heartbeat from core client for 30 sec - exiting 17:11:16 (4740): No heartbeat from core client for 30 sec - exiting 17:11:17 (4740): No heartbeat from core client for 30 sec - exiting 17:11:18 (4740): No heartbeat from core client for 30 sec - exiting 17:11:19 (4740): No heartbeat from core client for 30 sec - exiting 17:11:20 (4740): No heartbeat from core client for 30 sec - exiting 17:11:21 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:50:53 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:59 (7176): No heartbeat from core client for 30 sec - exiting 08:41:00 (7176): No heartbeat from core client for 30 sec - exiting 08:41:01 (7176): No heartbeat from core client for 30 sec - exiting 08:41:02 (7176): No heartbeat from core client for 30 sec - exiting 08:41:03 (7176): No heartbeat from core client for 30 sec - exiting 08:41:04 (7176): No heartbeat from core client for 30 sec - exiting 08:41:05 (7176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:52:13 (6260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:07:04 (7036): No heartbeat from core client for 30 sec - exiting 20:07:06 (7036): No heartbeat from core client for 30 sec - exiting 20:07:07 (7036): No heartbeat from core client for 30 sec - exiting 20:07:08 (7036): No heartbeat from core client for 30 sec - exiting 20:07:09 (7036): No heartbeat from core client for 30 sec - exiting 20:07:10 (7036): No heartbeat from core client for 30 sec - exiting 20:07:11 (7036): No heartbeat from core client for 30 sec - exiting 20:07:12 (7036): No heartbeat from core client for 30 sec - exiting 20:07:13 (7036): No heartbeat from core client for 30 sec - exiting 20:07:14 (7036): No heartbeat from core client for 30 sec - exiting 20:07:15 (7036): No heartbeat from core client for 30 sec - exiting 20:07:16 (7036): No heartbeat from core client for 30 sec - exiting 20:07:17 (7036): No heartbeat from core client for 30 sec - exiting 20:07:18 (7036): No heartbeat from core client for 30 sec - exiting 20:07:19 (7036): No heartbeat from core client for 30 sec - exiting 20:07:20 (7036): No heartbeat from core client for 30 sec - exiting 20:07:21 (7036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Dec 2011 21:04:46	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	933,120	1,071,814	1.1486
17 Dec 2011 07:08:32	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	907,200	1,041,418	1.1479
16 Dec 2011 20:55:08	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	881,280	1,010,208	1.1463
16 Dec 2011 11:42:29	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	855,360	979,558	1.1452
15 Dec 2011 18:20:18	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	829,440	948,073	1.1430
14 Dec 2011 16:55:24	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	803,520	918,577	1.1432
14 Dec 2011 00:10:24	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	777,600	887,271	1.1410
13 Dec 2011 04:29:26	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	751,680	856,893	1.1400
12 Dec 2011 18:43:15	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	725,760	826,649	1.1390
08 Dec 2011 14:19:35	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	699,840	794,988	1.1360
05 Dec 2011 18:09:26	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	673,920	764,593	1.1345
05 Dec 2011 00:54:03	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	648,000	733,268	1.1316
03 Dec 2011 16:18:10	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	622,080	702,191	1.1288
02 Dec 2011 18:56:05	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	596,160	673,235	1.1293
02 Dec 2011 11:02:32	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	570,240	644,780	1.1307
30 Nov 2011 17:28:00	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	544,320	615,972	1.1316
29 Nov 2011 13:21:52	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	518,400	586,156	1.1307
28 Nov 2011 07:05:05	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	492,480	556,862	1.1307
27 Nov 2011 14:52:19	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	466,560	526,622	1.1287
25 Nov 2011 16:58:02	1179386	13631485	hadcm3n_o6vw_1980_40_007545056_1	440,640	497,139	1.1282