Task 15862632

Name	hadcm3n_o1tt_2100_40_008270542_2
Workunit	8425666
Created	25 Jun 2013, 3:47:31 UTC
Sent	25 Jun 2013, 4:00:05 UTC
Report deadline	24 Sep 2013, 11:27:16 UTC
Received	21 Jul 2013, 21:30:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1176682
Run time	16 days 3 hours 43 min 39 sec
CPU time	14 days 15 hours 26 min 7 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x4308c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x4308c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2315004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2315000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7119,0xad0c82c0) malloc: * error for object 0x2311000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x10e1004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x10e1000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x10c5a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x10c5a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sun Jul 21 17:29:12 2013 hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c3204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c3804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c3c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c4004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c4404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c4804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c4e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c5204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c5604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c5a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c6004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c6404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c6804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c6c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c7004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c7604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c7a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c7e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c8204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c8604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c8a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c8e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c9204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c9604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20c9c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20ca204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20cac04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44501,0xad0c82c0) malloc: * error for object 0x20cac00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fac8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/0/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x9062c000 - 0x9065bfff /usr/lib/system/libsystem_info.dylib 0x91333000 - 0x91336fff /usr/lib/system/libmathCommon.A.dylib 0x9183e000 - 0x91846fff /usr/lib/system/liblaunch.dylib 0x91881000 - 0x9189ffff /usr/lib/system/libsystem_kernel.dylib 0x918a0000 - 0x918aefff /usr/lib/system/libdispatch.dylib 0x91ed1000 - 0x91ed5fff /usr/lib/system/libcache.dylib 0x91f29000 - 0x91f2afff /usr/lib/system/libquarantine.dylib 0x9224b000 - 0x92279fff /usr/lib/libSystem.B.dylib 0x92965000 - 0x9296dfff /usr/lib/system/libcopyfile.dylib 0x9317a000 - 0x9317cfff /usr/lib/system/libdyld.dylib 0x93e09000 - 0x93e10fff /usr/lib/system/libsystem_dnssd.dylib 0x947e2000 - 0x947eafff /usr/lib/system/libunwind.dylib 0x9483d000 - 0x94853fff /usr/lib/system/libxpc.dylib 0x9494e000 - 0x94957fff /usr/lib/libc++abi.dylib 0x949a3000 - 0x949a4fff /usr/lib/system/libsystem_sandbox.dylib 0x949a6000 - 0x949a7fff /usr/lib/system/libsystem_blocks.dylib 0x96a5f000 - 0x96a5ffff /usr/lib/system/libdnsinfo.dylib 0x96b79000 - 0x96bdbfff /usr/lib/libstdc++.6.dylib 0x96cdc000 - 0x96cddfff /usr/lib/system/libremovefile.dylib 0x971b0000 - 0x971b1fff /usr/lib/system/libunc.dylib 0x971b2000 - 0x971b6fff /usr/lib/system/libsystem_network.dylib 0x9884d000 - 0x98890fff /usr/lib/system/libcommonCrypto.dylib 0x9905d000 - 0x99064fff /usr/lib/system/libsystem_notify.dylib 0x99dc7000 - 0x99dccfff /usr/lib/system/libmacho.dylib 0x9a6bf000 - 0x9a6bffff /usr/lib/system/libkeymgr.dylib 0x9b510000 - 0x9b513fff /usr/lib/system/libcompiler_rt.dylib 0x9b514000 - 0x9b5dffff /usr/lib/system/libsystem_c.dylib 0x9cdec000 - 0x9cdfafff /usr/lib/libz.1.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2013 21:24:52	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	1,036,800	1,265,161	1.2203
23 Jul 2013 21:01:28	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	1,010,880	1,235,104	1.2218
23 Jul 2013 20:50:15	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	984,960	1,205,249	1.2237
23 Jul 2013 20:36:32	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	959,040	1,175,022	1.2252
23 Jul 2013 20:22:56	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	933,120	1,144,446	1.2265
23 Jul 2013 20:00:08	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	907,200	1,114,177	1.2281
23 Jul 2013 19:42:07	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	881,280	1,083,397	1.2293
23 Jul 2013 18:54:36	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	855,360	1,052,616	1.2306
23 Jul 2013 15:29:26	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	829,440	1,022,132	1.2323
23 Jul 2013 15:29:25	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	803,520	991,017	1.2333
23 Jul 2013 15:29:25	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	777,600	960,278	1.2349
23 Jul 2013 15:29:25	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	751,680	930,327	1.2377
23 Jul 2013 15:29:25	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	725,760	899,560	1.2395
10 Jul 2013 19:54:48	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	699,840	869,368	1.2422
10 Jul 2013 10:06:11	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	673,920	839,165	1.2452
09 Jul 2013 21:32:31	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	648,000	808,920	1.2483
09 Jul 2013 11:28:14	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	622,080	778,518	1.2515
08 Jul 2013 22:52:01	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	596,160	748,348	1.2553
08 Jul 2013 08:57:22	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	570,240	718,228	1.2595
07 Jul 2013 22:49:52	1176682	15862632	hadcm3n_o1tt_2100_40_008270542_2	544,320	687,875	1.2637