Task 15770816

Name	hadcm3n_n064_1960_40_008362144_0
Workunit	8513003
Created	10 May 2013, 10:28:00 UTC
Sent	10 May 2013, 10:28:16 UTC
Report deadline	9 Aug 2013, 17:55:27 UTC
Received	27 May 2013, 12:40:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1168218
Run time	11 days 4 hours 8 min 5 sec
CPU time	9 days 23 hours 46 min 5 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.14 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(12211,0xa04a9540) malloc: * error for object 0x1026c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12211,0xa04a9540) malloc: * error for object 0x1026c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12211,0xa04a9540) malloc: * error for object 0x5820a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12211,0xa04a9540) malloc: * error for object 0x5820a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(31693,0xa04a9540) malloc: * error for object 0x3001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Mon May 27 06:49:48 2013 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. 0 libSystem.B.dylib 0x947fcac5 small_malloc_from_free_list + 133 1 libSystem.B.dylib 0x947f94d9 szone_malloc_should_clear + 2271 2 libSystem.B.dylib 0x947f8ba8 malloc_zone_malloc + 81 3 libSystem.B.dylib 0x947f6c78 malloc + 50 4 libstdc++.6.dylib 0x96d27617 operator new(unsigned long) + 36 5 libstdc++.6.dylib 0x96d27703 operator new[](unsigned long) + 17 6 hadcm3n_6.07_i686-apple-darwin 0x0000b8e7 annual_cycle(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const, char const, int, int) + 3113 7 hadcm3n_6.07_i686-apple-darwin 0x0000d36b decadalMeans(int, char const) + 957 8 hadcm3n_6.07_i686-apple-darwin 0x000067ff doCM3Proc() + 185 9 hadcm3n_6.07_i686-apple-darwin 0x0000791c mainLoop() + 410 10 hadcm3n_6.07_i686-apple-darwin 0x000087c7 worker() + 2989 11 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 main + 491 12 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x947f4c0e mach_wait_until + 10 1 libSystem.B.dylib 0x9487c429 nanosleep + 345 2 libSystem.B.dylib 0x9487c2ca usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c boinc_sleep(double) + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 timer_thread(void) + 78 5 libSystem.B.dylib 0x94822259 _pthread_start + 345 6 libSystem.B.dylib 0x948220de thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8f658 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/11/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x947f4000 - 0x9499bfff /usr/lib/libSystem.B.dylib 0x949e1000 - 0x949effff /usr/lib/libz.1.dylib 0x95371000 - 0x95374fff /usr/lib/system/libmathCommon.A.dylib 0x96cdd000 - 0x96d47fff /usr/lib/libstdc++.6.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 May 2013 12:45:21	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	518,400	863,163	1.6651
26 May 2013 13:11:24	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	492,480	820,186	1.6654
25 May 2013 18:10:26	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	466,560	777,632	1.6667
24 May 2013 19:30:47	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	440,640	734,448	1.6668
23 May 2013 22:39:21	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	414,720	691,350	1.6670
23 May 2013 06:47:22	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	388,800	647,660	1.6658
22 May 2013 13:55:15	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	362,880	604,468	1.6658
21 May 2013 18:44:33	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	336,960	560,943	1.6647
20 May 2013 16:29:45	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	311,040	517,523	1.6638
19 May 2013 04:20:39	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	285,120	474,480	1.6641
18 May 2013 11:14:35	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	259,200	431,236	1.6637
17 May 2013 15:48:05	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	233,280	387,408	1.6607
16 May 2013 23:26:34	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	207,360	344,341	1.6606
16 May 2013 06:46:35	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	181,440	300,952	1.6587
15 May 2013 11:11:22	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	155,520	257,975	1.6588
14 May 2013 18:46:50	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	129,600	214,565	1.6556
13 May 2013 18:33:03	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	103,680	170,826	1.6476
13 May 2013 04:21:05	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	77,760	128,067	1.6470
12 May 2013 11:52:03	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	51,840	85,209	1.6437
11 May 2013 22:22:12	1168218	15770816	hadcm3n_n064_1960_40_008362144_0	25,920	42,568	1.6423