Task 19417726

Name	hadcm3n_ldov_198012_480_373_010412215_0
Workunit	10412215
Created	21 Mar 2016, 14:37:52 UTC
Sent	21 Mar 2016, 14:45:14 UTC
Report deadline	3 Mar 2017, 20:05:14 UTC
Received	19 Apr 2016, 21:18:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1120060
Run time	23 days 8 hours 1 min 45 sec
CPU time	19 days 7 hours 15 min 3 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.65 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:48:16 (572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(11430,0xa2dc3000) malloc: * error for object 0x381f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11430,0xa2dc3000) malloc: * error for object 0x383ac04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11430,0xa2dc3000) malloc: * error for object 0x383ac00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11430,0xa2dc3000) malloc: * error for object 0x483fa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(11430,0xa2dc3000) malloc: * error for object 0x847804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(17193,0xa2dc3000) malloc: * error for object 0x301dc00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(17943,0xa2dc3000) malloc: * error for object 0x2844604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17943,0xa2dc3000) malloc: * error for object 0x301f604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17943,0xa2dc3000) malloc: * error for object 0x4029e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17943,0xa2dc3000) malloc: * error for object 0x4029e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(17943,0xa2dc3000) malloc: * error for object 0x4029e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(19651,0xa2dc3000) malloc: * error for object 0x181de04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19651,0xa2dc3000) malloc: * error for object 0x181de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(19651,0xa2dc3000) malloc: * error for object 0x181de00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Tue Apr 19 16:59:17 2016 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fd98 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/14/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90011000 - 0x90013fff /usr/lib/system/libsystem_configuration.dylib 0x90014000 - 0x9001ffff /usr/lib/system/libcommonCrypto.dylib 0x90148000 - 0x90148fff /usr/lib/system/libsystem_blocks.dylib 0x9048a000 - 0x9048afff /usr/lib/system/libkeymgr.dylib 0x90647000 - 0x9064ffff /usr/lib/system/libsystem_pthread.dylib 0x911c0000 - 0x911c1fff /usr/lib/system/libremovefile.dylib 0x916c3000 - 0x916d1fff /usr/lib/libz.1.dylib 0x91861000 - 0x9187afff /usr/lib/system/libsystem_malloc.dylib 0x9187b000 - 0x91880fff /usr/lib/system/libmacho.dylib 0x91ac1000 - 0x91acbfff /usr/lib/system/libsystem_notify.dylib 0x92903000 - 0x9290bfff /usr/lib/system/libsystem_dnssd.dylib 0x9290c000 - 0x9293ffff /usr/lib/system/libsystem_m.dylib 0x9294c000 - 0x92950fff /usr/lib/system/libcache.dylib 0x92ab8000 - 0x92abbfff /usr/lib/system/libsystem_sandbox.dylib 0x92c90000 - 0x92cbafff /usr/lib/system/libdispatch.dylib 0x951b5000 - 0x951ccfff /usr/lib/system/libsystem_coretls.dylib 0x951cd000 - 0x951cefff /usr/lib/libSystem.B.dylib 0x96948000 - 0x96969fff /usr/lib/system/libsystem_kernel.dylib 0x969d3000 - 0x96a26fff /usr/lib/libstdc++.6.dylib 0x96eaf000 - 0x96eb1fff /usr/lib/system/libsystem_secinit.dylib 0x96f1f000 - 0x96f30fff /usr/lib/system/libsystem_trace.dylib 0x9772d000 - 0x97750fff /usr/lib/libc++abi.dylib 0x98bd2000 - 0x98bfcfff /usr/lib/system/libsystem_info.dylib 0x98bfd000 - 0x98c00fff /usr/lib/system/libdyld.dylib 0x98ecb000 - 0x98ee4fff /usr/lib/system/libsystem_asl.dylib 0x99221000 - 0x9928ffff /usr/lib/system/libsystem_network.dylib 0x999e9000 - 0x999e9fff /usr/lib/system/libunc.dylib 0x99b9d000 - 0x99c32fff /usr/lib/system/libsystem_c.dylib 0x99c42000 - 0x99c44fff /usr/lib/system/libquarantine.dylib 0x99e0c000 - 0x99e11fff /usr/lib/system/libcompiler_rt.dylib 0x9a174000 - 0x9a17dfff /usr/lib/system/libsystem_networkextension.dylib 0x9a38a000 - 0x9a393fff /usr/lib/system/libcopyfile.dylib 0x9ade1000 - 0x9ade7fff /usr/lib/system/libunwind.dylib 0x9ade8000 - 0x9ae4ffff /usr/lib/system/libcorecrypto.dylib 0x9ae50000 - 0x9ae52fff /usr/lib/system/libsystem_coreservices.dylib 0x9be57000 - 0x9be7ffff /usr/lib/system/libxpc.dylib 0x9c27c000 - 0x9c282fff /usr/lib/system/libsystem_platform.dylib 0x9ed82000 - 0x9ed82fff /usr/lib/system/liblaunch.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Apr 2016 16:10:38	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	1,036,800	1,667,039	1.6079
19 Apr 2016 00:52:35	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	1,010,880	1,626,708	1.6092
18 Apr 2016 10:39:09	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	984,960	1,586,643	1.6109
17 Apr 2016 20:02:18	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	959,040	1,546,200	1.6122
17 Apr 2016 05:04:30	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	933,120	1,506,040	1.6140
16 Apr 2016 13:50:48	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	907,200	1,466,066	1.6160
15 Apr 2016 18:56:15	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	881,280	1,425,377	1.6174
15 Apr 2016 12:16:39	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	855,360	1,385,579	1.6199
15 Apr 2016 12:14:48	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	829,440	1,347,149	1.6242
15 Apr 2016 12:12:37	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	803,520	1,308,237	1.6281
08 Apr 2016 23:55:44	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	777,600	1,271,077	1.6346
08 Apr 2016 18:02:33	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	751,680	1,236,921	1.6455
08 Apr 2016 00:24:13	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	725,760	1,202,042	1.6563
07 Apr 2016 12:33:43	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	699,840	1,167,202	1.6678
07 Apr 2016 00:45:58	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	673,920	1,132,123	1.6799
06 Apr 2016 11:00:09	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	648,000	1,090,020	1.6821
05 Apr 2016 21:23:57	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	622,080	1,047,755	1.6843
05 Apr 2016 07:55:28	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	596,160	1,005,736	1.6870
05 Apr 2016 00:18:06	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	570,240	963,464	1.6896
04 Apr 2016 01:45:05	1120060	19417726	hadcm3n_ldov_198012_480_373_010412215_0	544,320	920,802	1.6917