Task 16140108

Name	hadcm3n_o6a0_1940_40_008382888_4
Workunit	8533747
Created	9 Dec 2013, 12:54:30 UTC
Sent	9 Dec 2013, 12:55:16 UTC
Report deadline	10 Mar 2014, 20:22:27 UTC
Received	27 Dec 2013, 18:39:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1169268
Run time	12 days 20 hours 19 min
CPU time	12 days 10 hours 58 min 14 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.15 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x8023604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x8023600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2037204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2037200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x681f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x681f600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x2801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x7800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Fri Dec 27 09:55:21 2013 hadcm3n_6.07_i686-apple-darwin(90554,0xa00c1540) malloc: * error for object 0x5020604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. 0 libSystem.B.dylib 0x90630b0f small_free_list_remove_ptr + 246 1 libSystem.B.dylib 0x9062d5cc szone_free_definite_size + 3457 2 libSystem.B.dylib 0x9062c5e8 free + 244 3 hadcm3n_6.07_i686-apple-darwin 0x0000b99a annual_cycle(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const, char const, int, int) + 3292 4 hadcm3n_6.07_i686-apple-darwin 0x0000d36b decadalMeans(int, char const) + 957 5 hadcm3n_6.07_i686-apple-darwin 0x000067ff doCM3Proc() + 185 6 hadcm3n_6.07_i686-apple-darwin 0x0000791c mainLoop() + 410 7 hadcm3n_6.07_i686-apple-darwin 0x000087c7 worker() + 2989 8 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 main + 491 9 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x90625c0e mach_wait_until + 10 1 libSystem.B.dylib 0x906ad429 nanosleep + 345 2 libSystem.B.dylib 0x906ad2ca usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c boinc_sleep(double) + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 timer_thread(void) + 78 5 libSystem.B.dylib 0x90653259 _pthread_start + 345 6 libSystem.B.dylib 0x906530de thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8f6a8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/15/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90625000 - 0x907ccfff /usr/lib/libSystem.B.dylib 0x929e9000 - 0x92a53fff /usr/lib/libstdc++.6.dylib 0x93c3f000 - 0x93c42fff /usr/lib/system/libmathCommon.A.dylib 0x99951000 - 0x9995ffff /usr/lib/libz.1.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Dec 2013 16:10:22	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	777,600	1,076,290	1.3841
27 Dec 2013 00:19:49	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	751,680	1,040,988	1.3849
26 Dec 2013 10:54:17	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	725,760	1,005,683	1.3857
25 Dec 2013 23:21:28	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	699,840	970,190	1.3863
25 Dec 2013 03:44:28	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	673,920	934,655	1.3869
24 Dec 2013 17:41:07	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	648,000	899,311	1.3878
24 Dec 2013 06:53:11	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	622,080	863,880	1.3887
23 Dec 2013 18:49:42	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	596,160	828,472	1.3897
22 Dec 2013 06:59:01	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	570,240	792,802	1.3903
21 Dec 2013 19:21:12	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	544,320	757,403	1.3915
21 Dec 2013 09:11:28	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	518,400	722,029	1.3928
20 Dec 2013 15:37:17	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	492,480	686,739	1.3945
20 Dec 2013 01:58:53	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	466,560	651,332	1.3960
19 Dec 2013 09:45:33	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	440,640	615,991	1.3979
18 Dec 2013 21:12:02	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	414,720	580,560	1.3999
18 Dec 2013 10:47:28	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	388,800	545,182	1.4022
17 Dec 2013 14:00:06	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	362,880	509,594	1.4043
16 Dec 2013 22:33:01	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	336,960	474,193	1.4073
16 Dec 2013 11:50:52	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	311,040	438,600	1.4101
16 Dec 2013 00:36:08	1169268	16140108	hadcm3n_o6a0_1940_40_008382888_4	285,120	402,864	1.4130