Task 13772650

Name	hadcm3n_y8ie_1940_40_007615466_3
Workunit	7793596
Created	13 Dec 2011, 5:08:52 UTC
Sent	13 Dec 2011, 5:10:16 UTC
Report deadline	13 Mar 2012, 12:37:27 UTC
Received	18 Dec 2011, 4:38:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1244925
Run time	4 days 15 hours 4 min 34 sec
CPU time	3 days 22 hours 34 min 13 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.75 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x4034a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x4034a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x4034a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x833804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x4050004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x4050000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x2027604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x833804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x381f804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x381f800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x2800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x2801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(10006,0xa05c4540) malloc: * error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Sat Dec 17 19:56:40 2011 Thread 0 Crashed: Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10015, selfPID=10015, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(12683,0xa05c4540) malloc: * error for object 0x101c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12683,0xa05c4540) malloc: * error for object 0x282f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12683,0xa05c4540) malloc: * error for object 0x3829c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12683,0xa05c4540) malloc: * error for object 0x3829c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12683,0xa05c4540) malloc: * error for object 0x2802600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(12977,0xa05c4540) malloc: * error for object 0x401f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12977,0xa05c4540) malloc: * error for object 0x401f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12977,0xa05c4540) malloc: * error for object 0x481b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12977,0xa05c4540) malloc: * error for object 0x401f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12977,0xa05c4540) malloc: * error for object 0x382de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Sat Dec 17 23:32:10 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x95ce8b03 _small_free_list_remove_ptr + 234 1 libSystem.B.dylib 0x95ce55cc _szone_free_definite_size + 3457 2 libSystem.B.dylib 0x95ce45e8 _free + 244 3 hadcm3n_6.07_i686-apple-darwin 0x0000ba58 __Z12annual_cyclePKSt6vectorISsSaISsEEPKcii + 3482 4 hadcm3n_6.07_i686-apple-darwin 0x0000d36b __Z12decadalMeansiPKc + 957 5 hadcm3n_6.07_i686-apple-darwin 0x000067ff __Z9doCM3Procv + 185 6 hadcm3n_6.07_i686-apple-darwin 0x0000876a __Z6workerv + 2896 7 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 _main + 491 8 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 1: 0 libSystem.B.dylib 0x95cddc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x95d65429 _nanosleep + 345 2 libSystem.B.dylib 0x95d652ca _usleep + 61 3 hadcm3n_6.07_i686-apple-darwin 0x00071a7c __Z11boinc_sleepd + 188 4 hadcm3n_6.07_i686-apple-darwin 0x00067282 __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x95d0b259 __pthread_start + 345 6 libSystem.B.dylib 0x95d0b0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8f988 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/2/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x94424000 - 0x94432fff /usr/lib/libz.1.dylib 0x95cdd000 - 0x95e84fff /usr/lib/libSystem.B.dylib 0x98810000 - 0x9887afff /usr/lib/libstdc++.6.dylib 0x9a044000 - 0x9a047fff /usr/lib/system/libmathCommon.A.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Dec 2011 01:01:00	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	259,200	333,414	1.2863
17 Dec 2011 09:59:10	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	233,280	298,812	1.2809
16 Dec 2011 21:30:22	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	207,360	264,708	1.2766
16 Dec 2011 09:51:46	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	181,440	231,402	1.2754
15 Dec 2011 21:17:02	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	155,520	197,809	1.2719
15 Dec 2011 09:54:50	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	129,600	164,245	1.2673
14 Dec 2011 22:51:29	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	103,680	131,035	1.2638
14 Dec 2011 12:48:58	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	77,760	98,347	1.2648
14 Dec 2011 01:40:48	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	51,840	65,447	1.2625
13 Dec 2011 16:33:31	998485	13772650	hadcm3n_y8ie_1940_40_007615466_3	25,920	32,541	1.2554