Task 17257442

Name	hadcm3n_sd3l_1940_40_009115314_0
Workunit	9245650
Created	22 Oct 2014, 15:45:14 UTC
Sent	23 Oct 2014, 0:15:46 UTC
Report deadline	22 Jan 2015, 7:42:57 UTC
Received	19 Nov 2014, 19:31:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1300385
Run time	22 days 3 hours 13 min 16 sec
CPU time	17 days 17 hours 56 min 16 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	2.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x8b0a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x8b0a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x8b0a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x8b0a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x8b0a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x8b0a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x108c804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x108c800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x108c804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x108c800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x108c804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(35819,0xa08a51a8) malloc: * error for object 0x108c800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.5 build 13F34 Wed Nov 19 14:29:27 2014 Thread 0 Crashed: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4688 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x907b5000 - 0x907bbfff /usr/lib/system/libunwind.dylib 0x917ce000 - 0x917d6fff /usr/lib/system/libcopyfile.dylib 0x917f2000 - 0x917fafff /usr/lib/system/liblaunch.dylib 0x9180f000 - 0x91827fff /usr/lib/system/libsystem_malloc.dylib 0x91828000 - 0x9183afff /usr/lib/system/libsystem_asl.dylib 0x91c49000 - 0x91cdbfff /usr/lib/system/libsystem_c.dylib 0x920f8000 - 0x920f8fff /usr/lib/system/libkeymgr.dylib 0x920f9000 - 0x920fafff /usr/lib/system/libsystem_sandbox.dylib 0x9226c000 - 0x92275fff /usr/lib/system/libsystem_notify.dylib 0x92322000 - 0x9232dfff /usr/lib/system/libcommonCrypto.dylib 0x92c55000 - 0x92c80fff /usr/lib/system/libsystem_network.dylib 0x93839000 - 0x93851fff /usr/lib/system/libdispatch.dylib 0x93852000 - 0x93855fff /usr/lib/system/libdyld.dylib 0x94e4d000 - 0x94e7efff /usr/lib/system/libsystem_m.dylib 0x95b0b000 - 0x95b12fff /usr/lib/system/libsystem_pthread.dylib 0x95eef000 - 0x95ef0fff /usr/lib/system/libremovefile.dylib 0x960bc000 - 0x960c0fff /usr/lib/system/libcache.dylib 0x963f2000 - 0x96442fff /usr/lib/system/libcorecrypto.dylib 0x97014000 - 0x97022fff /usr/lib/libz.1.dylib 0x976a6000 - 0x976abfff /usr/lib/system/libsystem_platform.dylib 0x98e6b000 - 0x98e70fff /usr/lib/system/libcompiler_rt.dylib 0x991ce000 - 0x991cffff /usr/lib/libSystem.B.dylib 0x99e57000 - 0x99e5ffff /usr/lib/system/libsystem_dnssd.dylib 0x99f46000 - 0x99f48fff /usr/lib/system/libquarantine.dylib 0x99f49000 - 0x99f4afff /usr/lib/system/libsystem_blocks.dylib 0x9a8da000 - 0x9a8defff /usr/lib/system/libmacho.dylib 0x9b248000 - 0x9b265fff /usr/lib/system/libsystem_kernel.dylib 0x9bc3f000 - 0x9bc41fff /usr/lib/system/libsystem_configuration.dylib 0x9bd0c000 - 0x9bd30fff /usr/lib/system/libxpc.dylib 0x9bd31000 - 0x9bd59fff /usr/lib/system/libsystem_info.dylib 0x9be39000 - 0x9be3afff /usr/lib/system/libunc.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=70419, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.5 build 13F34 Wed Nov 19 14:29:39 2014 Thread 0 Crashed: Thread 1: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4688 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x907b5000 - 0x907bbfff /usr/lib/system/libunwind.dylib 0x917ce000 - 0x917d6fff /usr/lib/system/libcopyfile.dylib 0x917f2000 - 0x917fafff /usr/lib/system/liblaunch.dylib 0x9180f000 - 0x91827fff /usr/lib/system/libsystem_malloc.dylib 0x91828000 - 0x9183afff /usr/lib/system/libsystem_asl.dylib 0x91c49000 - 0x91cdbfff /usr/lib/system/libsystem_c.dylib 0x920f8000 - 0x920f8fff /usr/lib/system/libkeymgr.dylib 0x920f9000 - 0x920fafff /usr/lib/system/libsystem_sandbox.dylib 0x9226c000 - 0x92275fff /usr/lib/system/libsystem_notify.dylib 0x92322000 - 0x9232dfff /usr/lib/system/libcommonCrypto.dylib 0x92c55000 - 0x92c80fff /usr/lib/system/libsystem_network.dylib 0x93839000 - 0x93851fff /usr/lib/system/libdispatch.dylib 0x93852000 - 0x93855fff /usr/lib/system/libdyld.dylib 0x94e4d000 - 0x94e7efff /usr/lib/system/libsystem_m.dylib 0x95b0b000 - 0x95b12fff /usr/lib/system/libsystem_pthread.dylib 0x95eef000 - 0x95ef0fff /usr/lib/system/libremovefile.dylib 0x960bc000 - 0x960c0fff /usr/lib/system/libcache.dylib 0x963f2000 - 0x96442fff /usr/lib/system/libcorecrypto.dylib 0x97014000 - 0x97022fff /usr/lib/libz.1.dylib 0x976a6000 - 0x976abfff /usr/lib/system/libsystem_platform.dylib 0x98e6b000 - 0x98e70fff /usr/lib/system/libcompiler_rt.dylib 0x991ce000 - 0x991cffff /usr/lib/libSystem.B.dylib 0x99e57000 - 0x99e5ffff /usr/lib/system/libsystem_dnssd.dylib 0x99f46000 - 0x99f48fff /usr/lib/system/libquarantine.dylib 0x99f49000 - 0x99f4afff /usr/lib/system/libsystem_blocks.dylib 0x9a8da000 - 0x9a8defff /usr/lib/system/libmacho.dylib 0x9b248000 - 0x9b265fff /usr/lib/system/libsystem_kernel.dylib 0x9bc3f000 - 0x9bc41fff /usr/lib/system/libsystem_configuration.dylib 0x9bd0c000 - 0x9bd30fff /usr/lib/system/libxpc.dylib 0x9bd31000 - 0x9bd59fff /usr/lib/system/libsystem_info.dylib 0x9be39000 - 0x9be3afff /usr/lib/system/libunc.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=70419, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.5 build 13F34 Wed Nov 19 14:29:48 2014 Thread 0 Crashed: Thread 1: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4688 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x907b5000 - 0x907bbfff /usr/lib/system/libunwind.dylib 0x917ce000 - 0x917d6fff /usr/lib/system/libcopyfile.dylib 0x917f2000 - 0x917fafff /usr/lib/system/liblaunch.dylib 0x9180f000 - 0x91827fff /usr/lib/system/libsystem_malloc.dylib 0x91828000 - 0x9183afff /usr/lib/system/libsystem_asl.dylib 0x91c49000 - 0x91cdbfff /usr/lib/system/libsystem_c.dylib 0x920f8000 - 0x920f8fff /usr/lib/system/libkeymgr.dylib 0x920f9000 - 0x920fafff /usr/lib/system/libsystem_sandbox.dylib 0x9226c000 - 0x92275fff /usr/lib/system/libsystem_notify.dylib 0x92322000 - 0x9232dfff /usr/lib/system/libcommonCrypto.dylib 0x92c55000 - 0x92c80fff /usr/lib/system/libsystem_network.dylib 0x93839000 - 0x93851fff /usr/lib/system/libdispatch.dylib 0x93852000 - 0x93855fff /usr/lib/system/libdyld.dylib 0x94e4d000 - 0x94e7efff /usr/lib/system/libsystem_m.dylib 0x95b0b000 - 0x95b12fff /usr/lib/system/libsystem_pthread.dylib 0x95eef000 - 0x95ef0fff /usr/lib/system/libremovefile.dylib 0x960bc000 - 0x960c0fff /usr/lib/system/libcache.dylib 0x963f2000 - 0x96442fff /usr/lib/system/libcorecrypto.dylib 0x97014000 - 0x97022fff /usr/lib/libz.1.dylib 0x976a6000 - 0x976abfff /usr/lib/system/libsystem_platform.dylib 0x98e6b000 - 0x98e70fff /usr/lib/system/libcompiler_rt.dylib 0x991ce000 - 0x991cffff /usr/lib/libSystem.B.dylib 0x99e57000 - 0x99e5ffff /usr/lib/system/libsystem_dnssd.dylib 0x99f46000 - 0x99f48fff /usr/lib/system/libquarantine.dylib 0x99f49000 - 0x99f4afff /usr/lib/system/libsystem_blocks.dylib 0x9a8da000 - 0x9a8defff /usr/lib/system/libmacho.dylib 0x9b248000 - 0x9b265fff /usr/lib/system/libsystem_kernel.dylib 0x9bc3f000 - 0x9bc41fff /usr/lib/system/libsystem_configuration.dylib 0x9bd0c000 - 0x9bd30fff /usr/lib/system/libxpc.dylib 0x9bd31000 - 0x9bd59fff /usr/lib/system/libsystem_info.dylib 0x9be39000 - 0x9be3afff /usr/lib/system/libunc.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=70419, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.5 build 13F34 Wed Nov 19 14:29:57 2014 Thread 0 Crashed: Thread 1: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4688 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x907b5000 - 0x907bbfff /usr/lib/system/libunwind.dylib 0x917ce000 - 0x917d6fff /usr/lib/system/libcopyfile.dylib 0x917f2000 - 0x917fafff /usr/lib/system/liblaunch.dylib 0x9180f000 - 0x91827fff /usr/lib/system/libsystem_malloc.dylib 0x91828000 - 0x9183afff /usr/lib/system/libsystem_asl.dylib 0x91c49000 - 0x91cdbfff /usr/lib/system/libsystem_c.dylib 0x920f8000 - 0x920f8fff /usr/lib/system/libkeymgr.dylib 0x920f9000 - 0x920fafff /usr/lib/system/libsystem_sandbox.dylib 0x9226c000 - 0x92275fff /usr/lib/system/libsystem_notify.dylib 0x92322000 - 0x9232dfff /usr/lib/system/libcommonCrypto.dylib 0x92c55000 - 0x92c80fff /usr/lib/system/libsystem_network.dylib 0x93839000 - 0x93851fff /usr/lib/system/libdispatch.dylib 0x93852000 - 0x93855fff /usr/lib/system/libdyld.dylib 0x94e4d000 - 0x94e7efff /usr/lib/system/libsystem_m.dylib 0x95b0b000 - 0x95b12fff /usr/lib/system/libsystem_pthread.dylib 0x95eef000 - 0x95ef0fff /usr/lib/system/libremovefile.dylib 0x960bc000 - 0x960c0fff /usr/lib/system/libcache.dylib 0x963f2000 - 0x96442fff /usr/lib/system/libcorecrypto.dylib 0x97014000 - 0x97022fff /usr/lib/libz.1.dylib 0x976a6000 - 0x976abfff /usr/lib/system/libsystem_platform.dylib 0x98e6b000 - 0x98e70fff /usr/lib/system/libcompiler_rt.dylib 0x991ce000 - 0x991cffff /usr/lib/libSystem.B.dylib 0x99e57000 - 0x99e5ffff /usr/lib/system/libsystem_dnssd.dylib 0x99f46000 - 0x99f48fff /usr/lib/system/libquarantine.dylib 0x99f49000 - 0x99f4afff /usr/lib/system/libsystem_blocks.dylib 0x9a8da000 - 0x9a8defff /usr/lib/system/libmacho.dylib 0x9b248000 - 0x9b265fff /usr/lib/system/libsystem_kernel.dylib 0x9bc3f000 - 0x9bc41fff /usr/lib/system/libsystem_configuration.dylib 0x9bd0c000 - 0x9bd30fff /usr/lib/system/libxpc.dylib 0x9bd31000 - 0x9bd59fff /usr/lib/system/libsystem_info.dylib 0x9be39000 - 0x9be3afff /usr/lib/system/libunc.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=70419, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.5 build 13F34 Wed Nov 19 14:30:06 2014 Thread 0 Crashed: Thread 1: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4688 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x907b5000 - 0x907bbfff /usr/lib/system/libunwind.dylib 0x917ce000 - 0x917d6fff /usr/lib/system/libcopyfile.dylib 0x917f2000 - 0x917fafff /usr/lib/system/liblaunch.dylib 0x9180f000 - 0x91827fff /usr/lib/system/libsystem_malloc.dylib 0x91828000 - 0x9183afff /usr/lib/system/libsystem_asl.dylib 0x91c49000 - 0x91cdbfff /usr/lib/system/libsystem_c.dylib 0x920f8000 - 0x920f8fff /usr/lib/system/libkeymgr.dylib 0x920f9000 - 0x920fafff /usr/lib/system/libsystem_sandbox.dylib 0x9226c000 - 0x92275fff /usr/lib/system/libsystem_notify.dylib 0x92322000 - 0x9232dfff /usr/lib/system/libcommonCrypto.dylib 0x92c55000 - 0x92c80fff /usr/lib/system/libsystem_network.dylib 0x93839000 - 0x93851fff /usr/lib/system/libdispatch.dylib 0x93852000 - 0x93855fff /usr/lib/system/libdyld.dylib 0x94e4d000 - 0x94e7efff /usr/lib/system/libsystem_m.dylib 0x95b0b000 - 0x95b12fff /usr/lib/system/libsystem_pthread.dylib 0x95eef000 - 0x95ef0fff /usr/lib/system/libremovefile.dylib 0x960bc000 - 0x960c0fff /usr/lib/system/libcache.dylib 0x963f2000 - 0x96442fff /usr/lib/system/libcorecrypto.dylib 0x97014000 - 0x97022fff /usr/lib/libz.1.dylib 0x976a6000 - 0x976abfff /usr/lib/system/libsystem_platform.dylib 0x98e6b000 - 0x98e70fff /usr/lib/system/libcompiler_rt.dylib 0x991ce000 - 0x991cffff /usr/lib/libSystem.B.dylib 0x99e57000 - 0x99e5ffff /usr/lib/system/libsystem_dnssd.dylib 0x99f46000 - 0x99f48fff /usr/lib/system/libquarantine.dylib 0x99f49000 - 0x99f4afff /usr/lib/system/libsystem_blocks.dylib 0x9a8da000 - 0x9a8defff /usr/lib/system/libmacho.dylib 0x9b248000 - 0x9b265fff /usr/lib/system/libsystem_kernel.dylib 0x9bc3f000 - 0x9bc41fff /usr/lib/system/libsystem_configuration.dylib 0x9bd0c000 - 0x9bd30fff /usr/lib/system/libxpc.dylib 0x9bd31000 - 0x9bd59fff /usr/lib/system/libsystem_info.dylib 0x9be39000 - 0x9be3afff /usr/lib/system/libunc.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=70419, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.5 build 13F34 Wed Nov 19 14:30:15 2014 Thread 0 Crashed: Thread 1: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4688 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x907b5000 - 0x907bbfff /usr/lib/system/libunwind.dylib 0x917ce000 - 0x917d6fff /usr/lib/system/libcopyfile.dylib 0x917f2000 - 0x917fafff /usr/lib/system/liblaunch.dylib 0x9180f000 - 0x91827fff /usr/lib/system/libsystem_malloc.dylib 0x91828000 - 0x9183afff /usr/lib/system/libsystem_asl.dylib 0x91c49000 - 0x91cdbfff /usr/lib/system/libsystem_c.dylib 0x920f8000 - 0x920f8fff /usr/lib/system/libkeymgr.dylib 0x920f9000 - 0x920fafff /usr/lib/system/libsystem_sandbox.dylib 0x9226c000 - 0x92275fff /usr/lib/system/libsystem_notify.dylib 0x92322000 - 0x9232dfff /usr/lib/system/libcommonCrypto.dylib 0x92c55000 - 0x92c80fff /usr/lib/system/libsystem_network.dylib 0x93839000 - 0x93851fff /usr/lib/system/libdispatch.dylib 0x93852000 - 0x93855fff /usr/lib/system/libdyld.dylib 0x94e4d000 - 0x94e7efff /usr/lib/system/libsystem_m.dylib 0x95b0b000 - 0x95b12fff /usr/lib/system/libsystem_pthread.dylib 0x95eef000 - 0x95ef0fff /usr/lib/system/libremovefile.dylib 0x960bc000 - 0x960c0fff /usr/lib/system/libcache.dylib 0x963f2000 - 0x96442fff /usr/lib/system/libcorecrypto.dylib 0x97014000 - 0x97022fff /usr/lib/libz.1.dylib 0x976a6000 - 0x976abfff /usr/lib/system/libsystem_platform.dylib 0x98e6b000 - 0x98e70fff /usr/lib/system/libcompiler_rt.dylib 0x991ce000 - 0x991cffff /usr/lib/libSystem.B.dylib 0x99e57000 - 0x99e5ffff /usr/lib/system/libsystem_dnssd.dylib 0x99f46000 - 0x99f48fff /usr/lib/system/libquarantine.dylib 0x99f49000 - 0x99f4afff /usr/lib/system/libsystem_blocks.dylib 0x9a8da000 - 0x9a8defff /usr/lib/system/libmacho.dylib 0x9b248000 - 0x9b265fff /usr/lib/system/libsystem_kernel.dylib 0x9bc3f000 - 0x9bc41fff /usr/lib/system/libsystem_configuration.dylib 0x9bd0c000 - 0x9bd30fff /usr/lib/system/libxpc.dylib 0x9bd31000 - 0x9bd59fff /usr/lib/system/libsystem_info.dylib 0x9be39000 - 0x9be3afff /usr/lib/system/libunc.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=70419, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Nov 2014 06:20:36	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	855,360	1,503,342	1.7576
18 Nov 2014 09:16:53	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	829,440	1,457,782	1.7575
17 Nov 2014 14:48:23	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	803,520	1,408,699	1.7532
16 Nov 2014 13:55:18	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	777,600	1,358,626	1.7472
15 Nov 2014 22:42:58	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	751,680	1,313,708	1.7477
15 Nov 2014 01:39:15	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	725,760	1,268,720	1.7481
14 Nov 2014 09:53:46	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	699,840	1,222,673	1.7471
13 Nov 2014 17:29:19	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	673,920	1,176,165	1.7453
12 Nov 2014 15:26:22	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	648,000	1,131,173	1.7456
11 Nov 2014 22:10:19	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	622,080	1,084,889	1.7440
11 Nov 2014 03:57:52	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	596,160	1,038,178	1.7414
10 Nov 2014 09:17:24	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	570,240	992,468	1.7404
09 Nov 2014 11:27:02	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	544,320	946,901	1.7396
08 Nov 2014 18:43:00	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	518,400	901,360	1.7387
07 Nov 2014 20:06:37	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	492,480	856,413	1.7390
06 Nov 2014 16:07:31	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	466,560	809,526	1.7351
05 Nov 2014 18:17:26	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	440,640	762,895	1.7313
05 Nov 2014 01:45:15	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	414,720	716,336	1.7273
04 Nov 2014 06:38:35	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	388,800	671,470	1.7270
03 Nov 2014 11:35:37	1300385	17257442	hadcm3n_sd3l_1940_40_009115314_0	362,880	631,040	1.7390