Task 19419813

Name	hadcm3n_saxt_198012_480_375_010414160_0
Workunit	10414160
Created	21 Mar 2016, 16:32:09 UTC
Sent	22 Mar 2016, 15:37:08 UTC
Report deadline	4 Mar 2017, 20:57:08 UTC
Received	18 Apr 2016, 10:29:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1388508
Run time	17 days 10 hours 23 min 1 sec
CPU time	16 days 10 hours 45 min 58 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(16480,0xa351a000) malloc: * error for object 0x2822c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16480,0xa351a000) malloc: * error for object 0x2822c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16480,0xa351a000) malloc: * error for object 0x283e204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16480,0xa351a000) malloc: * error for object 0x283e200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x3044604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x182c204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x82d004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x2826204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x2826200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x2826204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x2826200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x2826204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x2826200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(20978,0xa351a000) malloc: * error for object 0x2826200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(98515,0xa351a000) malloc: * error for object 0x3808e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Mon Apr 18 05:29:03 2016 Thread 0 Crashed: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa98 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/23/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x127000 - 0x17afff /usr/lib/libstdc++.6.dylib 0x9075d000 - 0x90774fff /usr/lib/system/libsystem_coretls.dylib 0x90a47000 - 0x90a47fff /usr/lib/system/libsystem_blocks.dylib 0x91207000 - 0x9120afff /usr/lib/system/libdyld.dylib 0x9125e000 - 0x91262fff /usr/lib/system/libcache.dylib 0x91269000 - 0x9126efff /usr/lib/system/libcompiler_rt.dylib 0x92729000 - 0x9272bfff /usr/lib/system/libquarantine.dylib 0x935dc000 - 0x935eafff /usr/lib/libz.1.dylib 0x9411b000 - 0x9411efff /usr/lib/system/libsystem_sandbox.dylib 0x9411f000 - 0x94138fff /usr/lib/system/libsystem_asl.dylib 0x94455000 - 0x94457fff /usr/lib/system/libsystem_secinit.dylib 0x94a8d000 - 0x94aa6fff /usr/lib/system/libsystem_malloc.dylib 0x94b01000 - 0x94b29fff /usr/lib/system/libxpc.dylib 0x94c84000 - 0x94c8ffff /usr/lib/system/libcommonCrypto.dylib 0x94d50000 - 0x94d50fff /usr/lib/system/liblaunch.dylib 0x94d5c000 - 0x94d64fff /usr/lib/system/libsystem_dnssd.dylib 0x94dd0000 - 0x94dd6fff /usr/lib/system/libunwind.dylib 0x94ff3000 - 0x94ffbfff /usr/lib/system/libsystem_pthread.dylib 0x95c98000 - 0x95c99fff /usr/lib/libSystem.B.dylib 0x9677e000 - 0x96813fff /usr/lib/system/libsystem_c.dylib 0x96828000 - 0x9682afff /usr/lib/system/libsystem_configuration.dylib 0x98fcd000 - 0x99000fff /usr/lib/system/libsystem_m.dylib 0x999c3000 - 0x999d4fff /usr/lib/system/libsystem_trace.dylib 0x9a0f7000 - 0x9a0f7fff /usr/lib/system/libkeymgr.dylib 0x9ab48000 - 0x9ab4dfff /usr/lib/system/libmacho.dylib 0x9ab5b000 - 0x9ab5dfff /usr/lib/system/libsystem_coreservices.dylib 0x9bfaf000 - 0x9bfb5fff /usr/lib/system/libsystem_platform.dylib 0x9bfb9000 - 0x9bfbafff /usr/lib/system/libremovefile.dylib 0x9c077000 - 0x9c098fff /usr/lib/system/libsystem_kernel.dylib 0x9c412000 - 0x9c41cfff /usr/lib/system/libsystem_notify.dylib 0x9c5f6000 - 0x9c619fff /usr/lib/libc++abi.dylib 0x9cb7c000 - 0x9cbe3fff /usr/lib/system/libcorecrypto.dylib 0x9e01a000 - 0x9e044fff /usr/lib/system/libsystem_info.dylib 0x9e080000 - 0x9e0eefff /usr/lib/system/libsystem_network.dylib 0x9e2c7000 - 0x9e2f1fff /usr/lib/system/libdispatch.dylib 0x9ecb7000 - 0x9ecb7fff /usr/lib/system/libunc.dylib 0x9ed3b000 - 0x9ed44fff /usr/lib/system/libsystem_networkextension.dylib 0x9ed45000 - 0x9ed4efff /usr/lib/system/libcopyfile.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Apr 2016 09:31:16	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	777,600	1,421,154	1.8276
17 Apr 2016 16:27:54	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	751,680	1,374,171	1.8281
17 Apr 2016 00:38:31	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	725,760	1,326,313	1.8275
16 Apr 2016 06:55:52	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	699,840	1,278,600	1.8270
15 Apr 2016 14:29:23	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	673,920	1,231,101	1.8268
15 Apr 2016 12:16:13	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	648,000	1,183,149	1.8258
15 Apr 2016 12:14:00	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	622,080	1,137,124	1.8279
15 Apr 2016 12:11:34	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	596,160	1,089,814	1.8281
15 Apr 2016 12:08:40	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	570,240	1,041,999	1.8273
11 Apr 2016 00:43:40	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	544,320	993,745	1.8257
10 Apr 2016 08:34:14	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	518,400	946,123	1.8251
09 Apr 2016 09:57:12	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	492,480	898,974	1.8254
08 Apr 2016 19:52:05	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	466,560	851,415	1.8249
07 Apr 2016 14:00:06	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	440,640	803,934	1.8245
06 Apr 2016 19:26:08	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	414,720	756,813	1.8249
06 Apr 2016 01:35:05	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	388,800	709,405	1.8246
05 Apr 2016 03:10:24	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	362,880	662,501	1.8257
04 Apr 2016 05:51:38	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	336,960	615,321	1.8261
03 Apr 2016 13:20:45	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	311,040	567,933	1.8259
02 Apr 2016 15:24:41	1388508	19419813	hadcm3n_saxt_198012_480_375_010414160_0	285,120	519,966	1.8237