Task 16018348

Name	hadcm3n_7xcg_1980_40_008454499_1
Workunit	8605355
Created	15 Sep 2013, 21:25:02 UTC
Sent	15 Sep 2013, 22:31:31 UTC
Report deadline	16 Dec 2013, 5:58:42 UTC
Received	9 Oct 2013, 12:28:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1190785
Run time	13 days 0 hours 50 min 28 sec
CPU time	10 days 15 hours 56 min 7 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 08:50:43 (54028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:01 (54028): No heartbeat from core client for 30 sec - exiting 14:45:03 (58157): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/7xcgko.pji3c10 is not a valid UM file. Error converting file to netcdf: dataout/7xcgko.pji3c10 Error: Input file: dataout/7xcgko.pii3c10 is not a valid UM file. Error converting file to netcdf: dataout/7xcgko.pii3c10 Error: Input file: dataout/7xcgko.pfi3c10 is not a valid UM file. Error converting file to netcdf: dataout/7xcgko.pfi3c10 Error: Input file: dataout/7xcgka.phi3c10 is not a valid UM file. Error converting file to netcdf: dataout/7xcgka.phi3c10 Error: Input file: dataout/7xcgka.pgi3c10 is not a valid UM file. Error converting file to netcdf: dataout/7xcgka.pgi3c10 Error: Input file: dataout/7xcgka.pei3c10 is not a valid UM file. Error converting file to netcdf: dataout/7xcgka.pei3c10 Error: Input file: dataout/7xcgka.pdi3c10 is not a valid UM file. Error converting file to netcdf: dataout/7xcgka.pdi3c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(64921,0xac67e2c0) malloc: * error for object 0x20b4400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:01:08 (80764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:12:15 (25907): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:15:51 (48882): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:17:53 (48901): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:22:25 (48920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:25:38 (48946): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:27:46 (48972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:27:48 (48972): No heartbeat from core client for 30 sec - exiting 06:27:49 (48972): No heartbeat from core client for 30 sec - exiting 06:27:50 (48972): No heartbeat from core client for 30 sec - exiting 06:30:27 (48987): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:39:01 (49010): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:43:00 (49069): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:57:28 (49088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:00:25 (49171): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:05:19 (49190): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:10:03 (49224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:34:40 (49247): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:45:11 (49377): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:00:08 (49433): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:17 (49521): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:27:01 (49532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:35 (49664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:49:57 (49775): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:43 (49800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:19 (49825): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:59:48 (49850): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:02:40 (49869): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:21:42 (49890): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:52 (49991): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:53 (49991): No heartbeat from core client for 30 sec - exiting 09:23:54 (49991): No heartbeat from core client for 30 sec - exiting 09:25:58 (50002): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:37:57 (50024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:45 (50092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:47 (50112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:48:38 (50131): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:44 (50168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:02:50 (50201): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:12:15 (50253): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:13:53 (50310): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:16 (50319): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:25:05 (50374): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:25:07 (50374): No heartbeat from core client for 30 sec - exiting 10:25:08 (50374): No heartbeat from core client for 30 sec - exiting 10:25:09 (50374): No heartbeat from core client for 30 sec - exiting 10:25:10 (50374): No heartbeat from core client for 30 sec - exiting 10:35:06 (50389): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:35:12 (50389): No heartbeat from core client for 30 sec - exiting 10:37:50 (50458): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:37:51 (50458): No heartbeat from core client for 30 sec - exiting 10:37:52 (50458): No heartbeat from core client for 30 sec - exiting 10:37:53 (50458): No heartbeat from core client for 30 sec - exiting 10:37:54 (50458): No heartbeat from core client for 30 sec - exiting 10:42:23 (50475): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:46:23 (50512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:46:25 (50512): No heartbeat from core client for 30 sec - exiting 10:52:09 (50534): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:16 (50578): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:17 (50578): No heartbeat from core client for 30 sec - exiting 10:56:23 (50593): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:06:00 (50612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:10:05 (50673): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:10:07 (50673): No heartbeat from core client for 30 sec - exiting 11:20:33 (50693): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:00:20 (50752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:01:00 (50752): No heartbeat from core client for 30 sec - exiting 14:02:17 (52876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:25 (52899): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:13:19 (59163): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:12:53 (66070): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:45:37 (68992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:45:48 (68992): No heartbeat from core client for 30 sec - exiting 18:57:17 (71601): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(71677,0xac67e2c0) malloc: * error for object 0x2a35c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug 15:28:52 (71677): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:30:35 (19172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:32:56 (19185): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:35:24 (19197): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:26 (19216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:39:58 (19233): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:42:05 (19252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:44:02 (19263): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:47:24 (19277): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:49:41 (19303): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:51:28 (19319): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:53:17 (19331): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:55:14 (19344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:02:08 (19354): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:45 (19391): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:11:43 (19406): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:16:35 (19444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:32 (19471): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:35 (19498): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:26:30 (19510): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:51 (19529): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:23 (19542): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:33:54 (19560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:37:00 (19574): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:39:35 (19603): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:42:35 (19619): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:44:38 (19637): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:32 (19652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:48:12 (19664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:02 (19676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:53:00 (19692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:37 (19704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:57:48 (19722): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:53 (19734): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:02:34 (19748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:12:10 (19765): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:17:32 (19815): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:30 (19843): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:21:58 (19857): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:38 (19870): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:40 (19870): No heartbeat from core client for 30 sec - exiting 17:23:41 (19870): No heartbeat from core client for 30 sec - exiting 17:23:42 (19870): No heartbeat from core client for 30 sec - exiting 17:23:43 (19870): No heartbeat from core client for 30 sec - exiting 17:26:02 (19881): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:28:20 (19894): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:30:17 (19909): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:58 (19921): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:34:15 (19934): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:47 (19946): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:28 (19972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:49 (19981): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:34 (19996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:49:19 (20033): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:53:58 (20043): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:56:20 (20070): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:58:34 (20083): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:00:18 (20098): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:02:02 (20108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:04:03 (20121): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44242,0xac67e2c0) malloc: * error for object 0x9daa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(72285,0xac67e2c0) malloc: * error for object 0x11f2a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(72285,0xac67e2c0) malloc: * error for object 0x11f2a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(72285,0xac67e2c0) malloc: * error for object 0x21d1a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(72285,0xac67e2c0) malloc: * error for object 0x21d2800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Wed Oct 9 08:41:01 2013 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fda8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/1/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x900b6000 - 0x900b7fff /usr/lib/system/libsystem_sandbox.dylib 0x9096e000 - 0x9099dfff /usr/lib/system/libsystem_info.dylib 0x915a6000 - 0x915affff /usr/lib/libc++abi.dylib 0x91f73000 - 0x91fa1fff /usr/lib/libSystem.B.dylib 0x92413000 - 0x92413fff /usr/lib/system/libdnsinfo.dylib 0x93234000 - 0x93235fff /usr/lib/system/libsystem_blocks.dylib 0x9349b000 - 0x9349cfff /usr/lib/system/libquarantine.dylib 0x93ae7000 - 0x93ae7fff /usr/lib/system/libkeymgr.dylib 0x9501d000 - 0x950e8fff /usr/lib/system/libsystem_c.dylib 0x95464000 - 0x95469fff /usr/lib/system/libmacho.dylib 0x95655000 - 0x9565dfff /usr/lib/system/liblaunch.dylib 0x96031000 - 0x96039fff /usr/lib/system/libcopyfile.dylib 0x96233000 - 0x9623afff /usr/lib/system/libsystem_dnssd.dylib 0x96650000 - 0x96651fff /usr/lib/system/libunc.dylib 0x967ed000 - 0x9684ffff /usr/lib/libstdc++.6.dylib 0x97740000 - 0x97741fff /usr/lib/system/libremovefile.dylib 0x98f0c000 - 0x98f1afff /usr/lib/system/libdispatch.dylib 0x99012000 - 0x99016fff /usr/lib/system/libcache.dylib 0x99b39000 - 0x99b7cfff /usr/lib/system/libcommonCrypto.dylib 0x99bf1000 - 0x99c07fff /usr/lib/system/libxpc.dylib 0x99db9000 - 0x99dc0fff /usr/lib/system/libsystem_notify.dylib 0x99efd000 - 0x99f00fff /usr/lib/system/libmathCommon.A.dylib 0x9a1e1000 - 0x9a1e5fff /usr/lib/system/libsystem_network.dylib 0x9a1e6000 - 0x9a1e9fff /usr/lib/system/libcompiler_rt.dylib 0x9a34d000 - 0x9a36bfff /usr/lib/system/libsystem_kernel.dylib 0x9aed5000 - 0x9aed7fff /usr/lib/system/libdyld.dylib 0x9b6d8000 - 0x9b6e0fff /usr/lib/system/libunwind.dylib 0x9c09f000 - 0x9c0adfff /usr/lib/libz.1.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Oct 2013 15:47:57	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	1,036,800	921,028	0.8883
08 Oct 2013 08:33:58	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	1,010,880	898,718	0.8890
08 Oct 2013 01:19:28	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	984,960	876,365	0.8897
07 Oct 2013 17:43:48	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	959,040	854,002	0.8905
04 Oct 2013 12:33:36	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	933,120	833,277	0.8930
04 Oct 2013 03:33:35	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	907,200	810,553	0.8935
03 Oct 2013 18:43:32	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	881,280	787,849	0.8940
03 Oct 2013 09:46:46	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	855,360	765,150	0.8945
03 Oct 2013 00:52:34	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	829,440	742,440	0.8951
02 Oct 2013 16:01:07	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	803,520	719,706	0.8957
02 Oct 2013 07:53:59	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	777,600	696,774	0.8961
02 Oct 2013 00:13:52	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	751,680	673,492	0.8960
01 Oct 2013 16:35:04	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	725,760	650,215	0.8959
01 Oct 2013 08:10:30	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	699,840	626,969	0.8959
01 Oct 2013 00:32:07	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	673,920	603,738	0.8959
30 Sep 2013 16:52:08	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	648,000	580,552	0.8959
30 Sep 2013 10:25:58	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	622,080	557,695	0.8965
30 Sep 2013 01:01:42	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	596,160	535,115	0.8976
29 Sep 2013 17:17:32	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	570,240	511,773	0.8975
29 Sep 2013 09:33:58	1190785	16018348	hadcm3n_7xcg_1980_40_008454499_1	544,320	488,419	0.8973