Task 16075490

Name	hadcm3n_ofj1_1900_40_008475248_3
Workunit	8626087
Created	29 Oct 2013, 14:07:51 UTC
Sent	29 Oct 2013, 14:08:03 UTC
Report deadline	28 Jan 2014, 21:35:14 UTC
Received	16 Nov 2013, 0:46:19 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1176682
Run time	11 days 14 hours 27 min 12 sec
CPU time	10 days 13 hours 20 min 10 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.46 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(88156,0xaca4a2c0) malloc: * error for object 0x18b4c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x4405804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x5405804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x3bf0004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x3bf0000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x3bf0004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x3bf0000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0xbec004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0xbec000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x53eb004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x53ebe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x53ebe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x53ebe04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(91288,0xaca4a2c0) malloc: * error for object 0x53ebe00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/ofj1ko.pjc6c10 is not a valid UM file. Error converting file to netcdf: dataout/ofj1ko.pjc6c10 Error: Input file: dataout/ofj1ko.pic6c10 is not a valid UM file. Error converting file to netcdf: dataout/ofj1ko.pic6c10 Error: Input file: dataout/ofj1ko.pfc6c10 is not a valid UM file. Error converting file to netcdf: dataout/ofj1ko.pfc6c10 Error: Input file: dataout/ofj1ka.phc6c10 is not a valid UM file. Error converting file to netcdf: dataout/ofj1ka.phc6c10 Error: Input file: dataout/ofj1ka.pgc6c10 is not a valid UM file. Error converting file to netcdf: dataout/ofj1ka.pgc6c10 Error: Input file: dataout/ofj1ka.pec6c10 is not a valid UM file. Error converting file to netcdf: dataout/ofj1ka.pec6c10 Error: Input file: dataout/ofj1ka.pdc6c10 is not a valid UM file. Error converting file to netcdf: dataout/ofj1ka.pdc6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x3160e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x3160e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x3160e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x3160e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x2145804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x2160e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x2160e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x4955204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x946c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(33215,0xaca4a2c0) malloc: * error for object 0x3145804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(42162,0xaca4a2c0) malloc: * error for object 0x51b4804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(42162,0xaca4a2c0) malloc: * error for object 0x51b4800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(42162,0xaca4a2c0) malloc: * error for object 0x418d200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Fri Nov 15 17:03:35 2013 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fdd8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/6/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90139000 - 0x9014ffff /usr/lib/system/libxpc.dylib 0x90b63000 - 0x90b81fff /usr/lib/system/libsystem_kernel.dylib 0x90ec9000 - 0x90ef7fff /usr/lib/libSystem.B.dylib 0x9185c000 - 0x918befff /usr/lib/libstdc++.6.dylib 0x91b47000 - 0x91b47fff /usr/lib/system/libkeymgr.dylib 0x92a05000 - 0x92a06fff /usr/lib/system/libsystem_blocks.dylib 0x93f37000 - 0x93f7afff /usr/lib/system/libcommonCrypto.dylib 0x94460000 - 0x94465fff /usr/lib/system/libmacho.dylib 0x95ce0000 - 0x95ceefff /usr/lib/libz.1.dylib 0x96c6f000 - 0x96c76fff /usr/lib/system/libsystem_notify.dylib 0x96f07000 - 0x96f08fff /usr/lib/system/libquarantine.dylib 0x96f80000 - 0x96f83fff /usr/lib/system/libmathCommon.A.dylib 0x96fc1000 - 0x96fc4fff /usr/lib/system/libcompiler_rt.dylib 0x97a13000 - 0x97a42fff /usr/lib/system/libsystem_info.dylib 0x98661000 - 0x98669fff /usr/lib/system/liblaunch.dylib 0x9866a000 - 0x9866bfff /usr/lib/system/libremovefile.dylib 0x98dad000 - 0x98db5fff /usr/lib/system/libcopyfile.dylib 0x98e40000 - 0x98e4efff /usr/lib/system/libdispatch.dylib 0x98e4f000 - 0x98f1afff /usr/lib/system/libsystem_c.dylib 0x9933f000 - 0x99346fff /usr/lib/system/libsystem_dnssd.dylib 0x9b3fd000 - 0x9b3fefff /usr/lib/system/libsystem_sandbox.dylib 0x9b927000 - 0x9b929fff /usr/lib/system/libdyld.dylib 0x9b92a000 - 0x9b92afff /usr/lib/system/libdnsinfo.dylib 0x9c01b000 - 0x9c023fff /usr/lib/system/libunwind.dylib 0x9c046000 - 0x9c04afff /usr/lib/system/libcache.dylib 0x9c04b000 - 0x9c04cfff /usr/lib/system/libunc.dylib 0x9c08b000 - 0x9c08ffff /usr/lib/system/libsystem_network.dylib 0x9c6a4000 - 0x9c6adfff /usr/lib/libc++abi.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Nov 2013 18:26:28	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	777,600	909,815	1.1700
15 Nov 2013 09:25:54	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	751,680	879,867	1.1705
14 Nov 2013 20:23:05	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	725,760	849,448	1.1704
14 Nov 2013 11:09:03	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	699,840	819,428	1.1709
14 Nov 2013 00:51:09	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	673,920	789,572	1.1716
13 Nov 2013 15:48:57	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	648,000	759,593	1.1722
13 Nov 2013 06:15:45	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	622,080	728,851	1.1716
12 Nov 2013 19:58:03	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	596,160	698,350	1.1714
12 Nov 2013 11:32:33	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	570,240	668,489	1.1723
11 Nov 2013 22:19:27	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	544,320	638,300	1.1727
07 Nov 2013 04:24:28	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	518,400	608,158	1.1731
06 Nov 2013 18:36:16	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	492,480	577,688	1.1730
06 Nov 2013 09:17:46	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	466,560	547,517	1.1735
05 Nov 2013 20:48:36	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	440,640	516,351	1.1718
05 Nov 2013 18:42:45	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	414,720	487,343	1.1751
05 Nov 2013 03:46:39	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	388,800	457,115	1.1757
04 Nov 2013 18:22:24	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	362,880	426,627	1.1757
04 Nov 2013 08:54:26	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	336,960	396,281	1.1760
03 Nov 2013 23:30:59	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	311,040	365,790	1.1760
03 Nov 2013 13:14:11	1176682	16075490	hadcm3n_ofj1_1900_40_008475248_3	285,120	335,666	1.1773