Task 16130290

Name	hadcm3n_81zo_1980_40_008460519_4
Workunit	8611375
Created	4 Dec 2013, 10:53:55 UTC
Sent	4 Dec 2013, 10:56:17 UTC
Report deadline	5 Mar 2014, 18:23:28 UTC
Received	31 Jan 2014, 7:26:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1168753
Run time	16 days 11 hours 39 min 31 sec
CPU time	15 days 20 hours 25 min 14 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.15 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x6800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x6800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x3004e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x8800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x880f404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(62952,0xa0c2d540) malloc: * error for object 0x8813404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Thu Jan 30 21:21:34 2014 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. 0 libSystem.B.dylib 0x9a300b03 small_free_list_remove_ptr + 234 1 libSystem.B.dylib 0x9a2fd5cc szone_free_definite_size + 3457 2 libSystem.B.dylib 0x9a2fc5e8 free + 244 3 hadcm3n_6.07_i686-apple-darwin 0x00014206 std::_Vector_base<(anonymous namespace)::attribute, std::allocator<(anonymous namespace)::attribute> >::~_Vector_base() + 24 4 hadcm3n_6.07_i686-apple-darwin 0x000154a4 std::vector<(anonymous namespace)::attribute, std::allocator<(anonymous namespace)::attribute> >::~vector() + 110 5 hadcm3n_6.07_i686-apple-darwin 0x00013fb4 add_attributes(char const, char const) + 1700 6 hadcm3n_6.07_i686-apple-darwin 0x0000bb26 annual_cycle(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const, char const, int, int) + 3688 7 hadcm3n_6.07_i686-apple-darwin 0x0000d36b decadalMeans(int, char const) + 957 8 hadcm3n_6.07_i686-apple-darwin 0x000067ff doCM3Proc() + 185 9 hadcm3n_6.07_i686-apple-darwin 0x0000791c mainLoop() + 410 10 hadcm3n_6.07_i686-apple-darwin 0x000087c7 worker() + 2989 11 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 main + 491 12 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x9a2f5c0e mach_wait_until + 10 1 libSystem.B.dylib 0x9a37d429 nanosleep + 345 2 libSystem.B.dylib 0x9a37d2ca usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c boinc_sleep(double) + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 timer_thread(void) + 78 5 libSystem.B.dylib 0x9a323259 _pthread_start + 345 6 libSystem.B.dylib 0x9a3230de thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8f328 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/3/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x92149000 - 0x92157fff /usr/lib/libz.1.dylib 0x96500000 - 0x96503fff /usr/lib/system/libmathCommon.A.dylib 0x96574000 - 0x965defff /usr/lib/libstdc++.6.dylib 0x9a2f5000 - 0x9a49cfff /usr/lib/libSystem.B.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Jan 2014 03:22:13	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	777,600	1,369,483	1.7612
30 Jan 2014 17:38:00	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	751,680	1,322,690	1.7596
30 Jan 2014 17:38:00	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	725,760	1,275,860	1.7580
30 Jan 2014 17:38:00	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	699,840	1,228,884	1.7559
30 Jan 2014 17:38:00	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	673,920	1,181,914	1.7538
28 Jan 2014 06:10:46	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	648,000	1,134,884	1.7514
01 Jan 2014 20:17:00	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	622,080	1,089,441	1.7513
31 Dec 2013 21:56:40	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	596,160	1,045,221	1.7533
31 Dec 2013 09:14:52	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	570,240	1,000,905	1.7552
30 Dec 2013 11:02:30	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	544,320	956,592	1.7574
29 Dec 2013 22:04:25	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	518,400	911,282	1.7579
29 Dec 2013 00:30:50	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	492,480	866,008	1.7585
28 Dec 2013 11:11:09	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	466,560	819,569	1.7566
27 Dec 2013 21:58:02	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	440,640	774,144	1.7569
27 Dec 2013 08:56:27	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	414,720	728,547	1.7567
26 Dec 2013 19:38:38	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	388,800	683,206	1.7572
26 Dec 2013 06:38:18	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	362,880	637,948	1.7580
25 Dec 2013 17:19:02	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	336,960	592,160	1.7574
25 Dec 2013 04:19:36	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	311,040	547,117	1.7590
24 Dec 2013 15:20:30	1168753	16130290	hadcm3n_81zo_1980_40_008460519_4	285,120	501,994	1.7606