Task 15292738

Name	hadcm3n_zfnk_1880_40_008198547_3
Workunit	8353671
Created	20 Sep 2012, 7:19:55 UTC
Sent	20 Sep 2012, 7:20:22 UTC
Report deadline	20 Dec 2012, 14:47:33 UTC
Received	5 Oct 2012, 3:06:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1111289
Run time	3 days 14 hours 11 min 54 sec
CPU time	3 days 5 hours 35 min 42 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	3.56 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:12:13 (9106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:12:14 (9106): No heartbeat from core client for 30 sec - exiting 19:12:15 (9106): No heartbeat from core client for 30 sec - exiting 19:12:16 (9106): No heartbeat from core client for 30 sec - exiting 19:12:17 (9106): No heartbeat from core client for 30 sec - exiting 19:12:18 (9106): No heartbeat from core client for 30 sec - exiting 19:12:19 (9106): No heartbeat from core client for 30 sec - exiting 19:12:20 (9106): No heartbeat from core client for 30 sec - exiting 19:12:21 (9106): No heartbeat from core client for 30 sec - exiting 19:12:22 (9106): No heartbeat from core client for 30 sec - exiting 19:12:23 (9106): No heartbeat from core client for 30 sec - exiting 19:12:24 (9106): No heartbeat from core client for 30 sec - exiting 19:12:25 (9106): No heartbeat from core client for 30 sec - exiting 19:12:26 (9106): No heartbeat from core client for 30 sec - exiting 19:12:27 (9106): No heartbeat from core client for 30 sec - exiting 19:12:28 (9106): No heartbeat from core client for 30 sec - exiting 19:12:29 (9106): No heartbeat from core client for 30 sec - exiting 19:12:30 (9106): No heartbeat from core client for 30 sec - exiting 19:12:31 (9106): No heartbeat from core client for 30 sec - exiting 19:12:32 (9106): No heartbeat from core client for 30 sec - exiting 19:12:33 (9106): No heartbeat from core client for 30 sec - exiting 19:12:34 (9106): No heartbeat from core client for 30 sec - exiting 19:12:35 (9106): No heartbeat from core client for 30 sec - exiting 19:12:36 (9106): No heartbeat from core client for 30 sec - exiting 19:12:37 (9106): No heartbeat from core client for 30 sec - exiting 19:12:38 (9106): No heartbeat from core client for 30 sec - exiting 19:12:39 (9106): No heartbeat from core client for 30 sec - exiting 19:12:40 (9106): No heartbeat from core client for 30 sec - exiting 19:12:41 (9106): No heartbeat from core client for 30 sec - exiting 19:12:42 (9106): No heartbeat from core client for 30 sec - exiting 19:12:43 (9106): No heartbeat from core client for 30 sec - exiting 19:12:44 (9106): No heartbeat from core client for 30 sec - exiting 19:12:45 (9106): No heartbeat from core client for 30 sec - exiting 19:12:46 (9106): No heartbeat from core client for 30 sec - exiting 19:12:47 (9106): No heartbeat from core client for 30 sec - exiting 19:22:31 (12138): No heartbeat from core client for 30 sec - exiting 19:22:33 (12138): No heartbeat from core client for 30 sec - exiting 19:22:34 (12138): No heartbeat from core client for 30 sec - exiting 19:22:35 (12138): No heartbeat from core client for 30 sec - exiting 19:22:36 (12138): No heartbeat from core client for 30 sec - exiting 19:22:37 (12138): No heartbeat from core client for 30 sec - exiting 19:22:38 (12138): No heartbeat from core client for 30 sec - exiting 19:22:39 (12138): No heartbeat from core client for 30 sec - exiting 19:22:40 (12138): No heartbeat from core client for 30 sec - exiting 19:22:41 (12138): No heartbeat from core client for 30 sec - exiting 19:22:42 (12138): No heartbeat from core client for 30 sec - exiting 19:22:43 (12138): No heartbeat from core client for 30 sec - exiting 19:22:44 (12138): No heartbeat from core client for 30 sec - exiting 19:22:45 (12138): No heartbeat from core client for 30 sec - exiting 19:22:46 (12138): No heartbeat from core client for 30 sec - exiting 19:22:47 (12138): No heartbeat from core client for 30 sec - exiting 19:22:48 (12138): No heartbeat from core client for 30 sec - exiting 19:22:49 (12138): No heartbeat from core client for 30 sec - exiting 19:22:50 (12138): No heartbeat from core client for 30 sec - exiting 19:22:51 (12138): No heartbeat from core client for 30 sec - exiting 19:22:52 (12138): No heartbeat from core client for 30 sec - exiting 19:22:53 (12138): No heartbeat from core client for 30 sec - exiting 19:22:54 (12138): No heartbeat from core client for 30 sec - exiting 19:22:55 (12138): No heartbeat from core client for 30 sec - exiting 19:22:56 (12138): No heartbeat from core client for 30 sec - exiting 19:22:57 (12138): No heartbeat from core client for 30 sec - exiting 19:22:58 (12138): No heartbeat from core client for 30 sec - exiting 19:22:59 (12138): No heartbeat from core client for 30 sec - exiting 19:23:00 (12138): No heartbeat from core client for 30 sec - exiting 19:23:01 (12138): No heartbeat from core client for 30 sec - exiting 19:23:02 (12138): No heartbeat from core client for 30 sec - exiting 19:23:03 (12138): No heartbeat from core client for 30 sec - exiting 19:23:04 (12138): No heartbeat from core client for 30 sec - exiting 19:23:05 (12138): No heartbeat from core client for 30 sec - exiting 19:23:06 (12138): No heartbeat from core client for 30 sec - exiting 19:23:07 (12138): No heartbeat from core client for 30 sec - exiting 19:23:08 (12138): No heartbeat from core client for 30 sec - exiting 19:23:09 (12138): No heartbeat from core client for 30 sec - exiting 19:23:10 (12138): No heartbeat from core client for 30 sec - exiting 19:23:11 (12138): No heartbeat from core client for 30 sec - exiting 19:23:12 (12138): No heartbeat from core client for 30 sec - exiting 19:23:13 (12138): No heartbeat from core client for 30 sec - exiting 19:23:14 (12138): No heartbeat from core client for 30 sec - exiting 19:23:15 (12138): No heartbeat from core client for 30 sec - exiting 19:23:16 (12138): No heartbeat from core client for 30 sec - exiting 19:23:17 (12138): No heartbeat from core client for 30 sec - exiting 19:23:18 (12138): No heartbeat from core client for 30 sec - exiting 19:23:19 (12138): No heartbeat from core client for 30 sec - exiting 19:23:20 (12138): No heartbeat from core client for 30 sec - exiting 19:23:21 (12138): No heartbeat from core client for 30 sec - exiting 19:23:22 (12138): No heartbeat from core client for 30 sec - exiting 19:23:23 (12138): No heartbeat from core client for 30 sec - exiting 19:23:24 (12138): No heartbeat from core client for 30 sec - exiting 19:23:25 (12138): No heartbeat from core client for 30 sec - exiting 19:23:26 (12138): No heartbeat from core client for 30 sec - exiting 19:23:27 (12138): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Oct 3 19:46:44 2012 Thread 0 Crashed: CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Oct 3 20:15:27 2012 Thread 0 Crashed: CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 01:20:59 2012 Thread 0 Crashed: 0 libSystem.B.dylib 0x973cbc5a ___kill + 10 1 libSystem.B.dylib 0x9745e5a5 _raise + 26 2 libSystem.B.dylib 0x974746e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9736ac0e _mach_wait_until + 10 1 libSystem.B.dylib 0x973f2429 _nanosleep + 345 2 libSystem.B.dylib 0x973f22ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97398259 __pthread_start + 345 6 libSystem.B.dylib 0x973980de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc948 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92e28000 - 0x92e2bfff /usr/lib/system/libmathCommon.A.dylib 0x96e0b000 - 0x96e19fff /usr/lib/libz.1.dylib 0x9736a000 - 0x97511fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=241, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 01:21:02 2012 Thread 0 Crashed: CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 19:02:12 2012 Thread 0 Crashed: 0 libSystem.B.dylib 0x973cbc5a ___kill + 10 1 libSystem.B.dylib 0x9745e5a5 _raise + 26 2 libSystem.B.dylib 0x974746e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9736ac0e _mach_wait_until + 10 1 libSystem.B.dylib 0x973f2429 _nanosleep + 345 2 libSystem.B.dylib 0x973f22ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97398259 __pthread_start + 345 6 libSystem.B.dylib 0x973980de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc948 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92e28000 - 0x92e2bfff /usr/lib/system/libmathCommon.A.dylib 0x96e0b000 - 0x96e19fff /usr/lib/libz.1.dylib 0x9736a000 - 0x97511fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18080, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 19:02:15 2012 Thread 0 Crashed: 0 libSystem.B.dylib 0x973cbc5a ___kill + 10 1 libSystem.B.dylib 0x9745e5a5 _raise + 26 2 libSystem.B.dylib 0x974746e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9736ac0e _mach_wait_until + 10 1 libSystem.B.dylib 0x973f2429 _nanosleep + 345 2 libSystem.B.dylib 0x973f22ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97398259 __pthread_start + 345 6 libSystem.B.dylib 0x973980de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc948 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92e28000 - 0x92e2bfff /usr/lib/system/libmathCommon.A.dylib 0x96e0b000 - 0x96e19fff /usr/lib/libz.1.dylib 0x9736a000 - 0x97511fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18080, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 19:02:18 2012 Thread 0 Crashed: 0 libSystem.B.dylib 0x973cbc5a ___kill + 10 1 libSystem.B.dylib 0x9745e5a5 _raise + 26 2 libSystem.B.dylib 0x974746e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9736ac0e _mach_wait_until + 10 1 libSystem.B.dylib 0x973f2429 _nanosleep + 345 2 libSystem.B.dylib 0x973f22ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97398259 __pthread_start + 345 6 libSystem.B.dylib 0x973980de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc948 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92e28000 - 0x92e2bfff /usr/lib/system/libmathCommon.A.dylib 0x96e0b000 - 0x96e19fff /usr/lib/libz.1.dylib 0x9736a000 - 0x97511fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18080, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 19:02:20 2012 Thread 0 Crashed: 0 libSystem.B.dylib 0x973cbc5a ___kill + 10 1 libSystem.B.dylib 0x9745e5a5 _raise + 26 2 libSystem.B.dylib 0x974746e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9736ac0e _mach_wait_until + 10 1 libSystem.B.dylib 0x973f2429 _nanosleep + 345 2 libSystem.B.dylib 0x973f22ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97398259 __pthread_start + 345 6 libSystem.B.dylib 0x973980de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc948 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92e28000 - 0x92e2bfff /usr/lib/system/libmathCommon.A.dylib 0x96e0b000 - 0x96e19fff /usr/lib/libz.1.dylib 0x9736a000 - 0x97511fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18080, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 19:02:23 2012 Thread 0 Crashed: CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Oct 4 23:04:05 2012 Thread 0 Crashed: 0 libSystem.B.dylib 0x973cbc5a ___kill + 10 1 libSystem.B.dylib 0x9745e5a5 _raise + 26 2 libSystem.B.dylib 0x974746e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9736ac0e _mach_wait_until + 10 1 libSystem.B.dylib 0x973f2429 _nanosleep + 345 2 libSystem.B.dylib 0x973f22ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x97398259 __pthread_start + 345 6 libSystem.B.dylib 0x973980de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc948 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92e28000 - 0x92e2bfff /usr/lib/system/libmathCommon.A.dylib 0x96e0b000 - 0x96e19fff /usr/lib/libz.1.dylib 0x9736a000 - 0x97511fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20177, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Oct 2012 20:04:24	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	207,360	270,092	1.3025
03 Oct 2012 07:56:19	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	181,440	239,802	1.3217
02 Oct 2012 15:53:30	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	155,520	206,073	1.3251
30 Sep 2012 11:32:04	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	129,600	168,204	1.2979
29 Sep 2012 23:59:46	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	103,680	134,547	1.2977
29 Sep 2012 09:36:01	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	77,760	101,242	1.3020
28 Sep 2012 14:17:28	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	51,840	65,103	1.2558
27 Sep 2012 22:33:51	1111289	15292738	hadcm3n_zfnk_1880_40_008198547_3	25,920	32,619	1.2584