Task 15278839

Name	hadcm3n_za3j_1880_40_008200203_1
Workunit	8355327
Created	13 Sep 2012, 5:28:37 UTC
Sent	14 Sep 2012, 6:16:40 UTC
Report deadline	14 Dec 2012, 13:43:51 UTC
Received	12 Oct 2012, 0:15:19 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1289096
Run time	10 days 0 hours 31 min 13 sec
CPU time	9 days 2 hours 8 min 14 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:17:17 (45890): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:17:18 (45890): No heartbeat from core client for 30 sec - exiting 17:17:19 (45890): No heartbeat from core client for 30 sec - exiting 17:17:20 (45890): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:25:10 (58019): No heartbeat from core client for 30 sec - exiting 08:25:11 (58019): No heartbeat from core client for 30 sec - exiting 08:25:12 (58019): No heartbeat from core client for 30 sec - exiting 08:25:13 (58019): No heartbeat from core client for 30 sec - exiting 08:25:14 (58019): No heartbeat from core client for 30 sec - exiting 08:25:15 (58019): No heartbeat from core client for 30 sec - exiting 08:25:16 (58019): No heartbeat from core client for 30 sec - exiting 08:25:17 (58019): No heartbeat from core client for 30 sec - exiting 08:25:18 (58019): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:38:14 (38445): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:20:16 (60197): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:11:12 (81742): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:38:13 (5479): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:27:32 (20320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(28583,0xacbef2c0) malloc: * error for object 0x893a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28583,0xacbef2c0) malloc: * error for object 0x18a8c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(28583,0xacbef2c0) malloc: * error for object 0x18a8c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x33bc004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x33bc000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x3bbc004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x3bbc000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x3bbc004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x3bbc000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x3bbc004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x3bbc000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(40878,0xacbef2c0) malloc: * error for object 0x3bbc000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(47746,0xacbef2c0) malloc: * error for object 0x33eaa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(47746,0xacbef2c0) malloc: * error for object 0x43e9a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Thu Oct 11 17:13:25 2012 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fcf8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/8/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x954e9000 - 0x9554bfff /usr/lib/libstdc++.6.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x99325000 - 0x9932efff /usr/lib/libc++abi.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Oct 2012 09:20:47	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	518,400	769,393	1.4842
10 Oct 2012 18:40:20	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	492,480	731,186	1.4847
09 Oct 2012 08:05:23	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	466,560	693,837	1.4871
04 Oct 2012 13:04:06	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	440,640	656,440	1.4897
04 Oct 2012 01:51:36	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	414,720	619,056	1.4927
02 Oct 2012 08:16:58	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	388,800	579,374	1.4902
01 Oct 2012 16:12:05	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	362,880	534,975	1.4742
28 Sep 2012 12:07:01	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	336,960	497,080	1.4752
28 Sep 2012 00:24:21	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	311,040	459,582	1.4776
27 Sep 2012 12:45:01	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	285,120	422,192	1.4808
27 Sep 2012 01:07:15	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	259,200	384,211	1.4823
26 Sep 2012 12:24:29	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	233,280	347,000	1.4875
25 Sep 2012 20:59:19	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	207,360	309,360	1.4919
25 Sep 2012 08:32:55	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	181,440	272,375	1.5012
24 Sep 2012 21:25:09	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	155,520	235,613	1.5150
20 Sep 2012 07:06:19	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	129,600	194,436	1.5003
19 Sep 2012 11:53:53	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	103,680	152,492	1.4708
19 Sep 2012 01:01:06	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	77,760	115,824	1.4895
18 Sep 2012 12:26:17	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	51,840	77,365	1.4924
17 Sep 2012 21:22:35	1216073	15278839	hadcm3n_za3j_1880_40_008200203_1	25,920	38,306	1.4779