Task 17324175

Name	hadcm3n_s19g_1940_40_009094797_1
Workunit	9225133
Created	30 Oct 2014, 7:35:12 UTC
Sent	30 Oct 2014, 7:35:41 UTC
Report deadline	29 Jan 2015, 15:02:52 UTC
Received	17 Nov 2014, 6:53:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1319483
Run time	9 days 0 hours 46 min 37 sec
CPU time	8 days 2 hours 33 min 24 sec
Validate state	Invalid
Credit	3,732.48
Device peak FLOPS	2.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(52800,0xac1242c0) malloc: * error for object 0x13d0804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=52803, selfPID=52803, iMonCtr=1 hadcm3n_6.07_i686-apple-darwin(54251,0xac1242c0) malloc: * error for object 0x1342604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(54251,0xac1242c0) malloc: * error for object 0x1342600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(54251,0xac1242c0) malloc: * error for object 0xb41804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(54251,0xac1242c0) malloc: * error for object 0xb41800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(54251,0xac1242c0) malloc: * error for object 0xb41804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55401,0xac1242c0) malloc: * error for object 0x95ec00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sun Nov 16 23:48:27 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90213000 - 0x90214fff /usr/lib/system/libsystem_blocks.dylib 0x90a16000 - 0x90a19fff /usr/lib/system/libmathCommon.A.dylib 0x90bdc000 - 0x90ca7fff /usr/lib/system/libsystem_c.dylib 0x91c77000 - 0x91c85fff /usr/lib/system/libdispatch.dylib 0x91d8b000 - 0x91d8ffff /usr/lib/system/libsystem_network.dylib 0x91d90000 - 0x91d90fff /usr/lib/system/libkeymgr.dylib 0x91dd2000 - 0x91de0fff /usr/lib/libz.1.dylib 0x92d76000 - 0x92d77fff /usr/lib/system/libunc.dylib 0x930af000 - 0x930b7fff /usr/lib/system/libcopyfile.dylib 0x9374a000 - 0x93751fff /usr/lib/system/libsystem_dnssd.dylib 0x937ef000 - 0x937f0fff /usr/lib/system/libquarantine.dylib 0x93c3a000 - 0x93c42fff /usr/lib/system/liblaunch.dylib 0x93c4f000 - 0x93c56fff /usr/lib/system/libsystem_notify.dylib 0x93c64000 - 0x93c68fff /usr/lib/system/libcache.dylib 0x93dfa000 - 0x93e10fff /usr/lib/system/libxpc.dylib 0x94dcc000 - 0x94dfafff /usr/lib/libSystem.B.dylib 0x94dfb000 - 0x94e3efff /usr/lib/system/libcommonCrypto.dylib 0x95b60000 - 0x95b61fff /usr/lib/system/libremovefile.dylib 0x95d3a000 - 0x95d3dfff /usr/lib/system/libcompiler_rt.dylib 0x962c4000 - 0x962c6fff /usr/lib/system/libdyld.dylib 0x99cc4000 - 0x99cc9fff /usr/lib/system/libmacho.dylib 0x9ac13000 - 0x9ac14fff /usr/lib/system/libsystem_sandbox.dylib 0x9bb98000 - 0x9bbc7fff /usr/lib/system/libsystem_info.dylib 0x9bf9b000 - 0x9bf9bfff /usr/lib/system/libdnsinfo.dylib 0x9bfa0000 - 0x9bfbefff /usr/lib/system/libsystem_kernel.dylib 0x9c217000 - 0x9c21ffff /usr/lib/system/libunwind.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=79248, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sun Nov 16 23:48:30 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90213000 - 0x90214fff /usr/lib/system/libsystem_blocks.dylib 0x90a16000 - 0x90a19fff /usr/lib/system/libmathCommon.A.dylib 0x90bdc000 - 0x90ca7fff /usr/lib/system/libsystem_c.dylib 0x91c77000 - 0x91c85fff /usr/lib/system/libdispatch.dylib 0x91d8b000 - 0x91d8ffff /usr/lib/system/libsystem_network.dylib 0x91d90000 - 0x91d90fff /usr/lib/system/libkeymgr.dylib 0x91dd2000 - 0x91de0fff /usr/lib/libz.1.dylib 0x92d76000 - 0x92d77fff /usr/lib/system/libunc.dylib 0x930af000 - 0x930b7fff /usr/lib/system/libcopyfile.dylib 0x9374a000 - 0x93751fff /usr/lib/system/libsystem_dnssd.dylib 0x937ef000 - 0x937f0fff /usr/lib/system/libquarantine.dylib 0x93c3a000 - 0x93c42fff /usr/lib/system/liblaunch.dylib 0x93c4f000 - 0x93c56fff /usr/lib/system/libsystem_notify.dylib 0x93c64000 - 0x93c68fff /usr/lib/system/libcache.dylib 0x93dfa000 - 0x93e10fff /usr/lib/system/libxpc.dylib 0x94dcc000 - 0x94dfafff /usr/lib/libSystem.B.dylib 0x94dfb000 - 0x94e3efff /usr/lib/system/libcommonCrypto.dylib 0x95b60000 - 0x95b61fff /usr/lib/system/libremovefile.dylib 0x95d3a000 - 0x95d3dfff /usr/lib/system/libcompiler_rt.dylib 0x962c4000 - 0x962c6fff /usr/lib/system/libdyld.dylib 0x99cc4000 - 0x99cc9fff /usr/lib/system/libmacho.dylib 0x9ac13000 - 0x9ac14fff /usr/lib/system/libsystem_sandbox.dylib 0x9bb98000 - 0x9bbc7fff /usr/lib/system/libsystem_info.dylib 0x9bf9b000 - 0x9bf9bfff /usr/lib/system/libdnsinfo.dylib 0x9bfa0000 - 0x9bfbefff /usr/lib/system/libsystem_kernel.dylib 0x9c217000 - 0x9c21ffff /usr/lib/system/libunwind.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=79248, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sun Nov 16 23:48:33 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90213000 - 0x90214fff /usr/lib/system/libsystem_blocks.dylib 0x90a16000 - 0x90a19fff /usr/lib/system/libmathCommon.A.dylib 0x90bdc000 - 0x90ca7fff /usr/lib/system/libsystem_c.dylib 0x91c77000 - 0x91c85fff /usr/lib/system/libdispatch.dylib 0x91d8b000 - 0x91d8ffff /usr/lib/system/libsystem_network.dylib 0x91d90000 - 0x91d90fff /usr/lib/system/libkeymgr.dylib 0x91dd2000 - 0x91de0fff /usr/lib/libz.1.dylib 0x92d76000 - 0x92d77fff /usr/lib/system/libunc.dylib 0x930af000 - 0x930b7fff /usr/lib/system/libcopyfile.dylib 0x9374a000 - 0x93751fff /usr/lib/system/libsystem_dnssd.dylib 0x937ef000 - 0x937f0fff /usr/lib/system/libquarantine.dylib 0x93c3a000 - 0x93c42fff /usr/lib/system/liblaunch.dylib 0x93c4f000 - 0x93c56fff /usr/lib/system/libsystem_notify.dylib 0x93c64000 - 0x93c68fff /usr/lib/system/libcache.dylib 0x93dfa000 - 0x93e10fff /usr/lib/system/libxpc.dylib 0x94dcc000 - 0x94dfafff /usr/lib/libSystem.B.dylib 0x94dfb000 - 0x94e3efff /usr/lib/system/libcommonCrypto.dylib 0x95b60000 - 0x95b61fff /usr/lib/system/libremovefile.dylib 0x95d3a000 - 0x95d3dfff /usr/lib/system/libcompiler_rt.dylib 0x962c4000 - 0x962c6fff /usr/lib/system/libdyld.dylib 0x99cc4000 - 0x99cc9fff /usr/lib/system/libmacho.dylib 0x9ac13000 - 0x9ac14fff /usr/lib/system/libsystem_sandbox.dylib 0x9bb98000 - 0x9bbc7fff /usr/lib/system/libsystem_info.dylib 0x9bf9b000 - 0x9bf9bfff /usr/lib/system/libdnsinfo.dylib 0x9bfa0000 - 0x9bfbefff /usr/lib/system/libsystem_kernel.dylib 0x9c217000 - 0x9c21ffff /usr/lib/system/libunwind.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=79248, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sun Nov 16 23:48:36 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90213000 - 0x90214fff /usr/lib/system/libsystem_blocks.dylib 0x90a16000 - 0x90a19fff /usr/lib/system/libmathCommon.A.dylib 0x90bdc000 - 0x90ca7fff /usr/lib/system/libsystem_c.dylib 0x91c77000 - 0x91c85fff /usr/lib/system/libdispatch.dylib 0x91d8b000 - 0x91d8ffff /usr/lib/system/libsystem_network.dylib 0x91d90000 - 0x91d90fff /usr/lib/system/libkeymgr.dylib 0x91dd2000 - 0x91de0fff /usr/lib/libz.1.dylib 0x92d76000 - 0x92d77fff /usr/lib/system/libunc.dylib 0x930af000 - 0x930b7fff /usr/lib/system/libcopyfile.dylib 0x9374a000 - 0x93751fff /usr/lib/system/libsystem_dnssd.dylib 0x937ef000 - 0x937f0fff /usr/lib/system/libquarantine.dylib 0x93c3a000 - 0x93c42fff /usr/lib/system/liblaunch.dylib 0x93c4f000 - 0x93c56fff /usr/lib/system/libsystem_notify.dylib 0x93c64000 - 0x93c68fff /usr/lib/system/libcache.dylib 0x93dfa000 - 0x93e10fff /usr/lib/system/libxpc.dylib 0x94dcc000 - 0x94dfafff /usr/lib/libSystem.B.dylib 0x94dfb000 - 0x94e3efff /usr/lib/system/libcommonCrypto.dylib 0x95b60000 - 0x95b61fff /usr/lib/system/libremovefile.dylib 0x95d3a000 - 0x95d3dfff /usr/lib/system/libcompiler_rt.dylib 0x962c4000 - 0x962c6fff /usr/lib/system/libdyld.dylib 0x99cc4000 - 0x99cc9fff /usr/lib/system/libmacho.dylib 0x9ac13000 - 0x9ac14fff /usr/lib/system/libsystem_sandbox.dylib 0x9bb98000 - 0x9bbc7fff /usr/lib/system/libsystem_info.dylib 0x9bf9b000 - 0x9bf9bfff /usr/lib/system/libdnsinfo.dylib 0x9bfa0000 - 0x9bfbefff /usr/lib/system/libsystem_kernel.dylib 0x9c217000 - 0x9c21ffff /usr/lib/system/libunwind.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=79248, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sun Nov 16 23:48:39 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90213000 - 0x90214fff /usr/lib/system/libsystem_blocks.dylib 0x90a16000 - 0x90a19fff /usr/lib/system/libmathCommon.A.dylib 0x90bdc000 - 0x90ca7fff /usr/lib/system/libsystem_c.dylib 0x91c77000 - 0x91c85fff /usr/lib/system/libdispatch.dylib 0x91d8b000 - 0x91d8ffff /usr/lib/system/libsystem_network.dylib 0x91d90000 - 0x91d90fff /usr/lib/system/libkeymgr.dylib 0x91dd2000 - 0x91de0fff /usr/lib/libz.1.dylib 0x92d76000 - 0x92d77fff /usr/lib/system/libunc.dylib 0x930af000 - 0x930b7fff /usr/lib/system/libcopyfile.dylib 0x9374a000 - 0x93751fff /usr/lib/system/libsystem_dnssd.dylib 0x937ef000 - 0x937f0fff /usr/lib/system/libquarantine.dylib 0x93c3a000 - 0x93c42fff /usr/lib/system/liblaunch.dylib 0x93c4f000 - 0x93c56fff /usr/lib/system/libsystem_notify.dylib 0x93c64000 - 0x93c68fff /usr/lib/system/libcache.dylib 0x93dfa000 - 0x93e10fff /usr/lib/system/libxpc.dylib 0x94dcc000 - 0x94dfafff /usr/lib/libSystem.B.dylib 0x94dfb000 - 0x94e3efff /usr/lib/system/libcommonCrypto.dylib 0x95b60000 - 0x95b61fff /usr/lib/system/libremovefile.dylib 0x95d3a000 - 0x95d3dfff /usr/lib/system/libcompiler_rt.dylib 0x962c4000 - 0x962c6fff /usr/lib/system/libdyld.dylib 0x99cc4000 - 0x99cc9fff /usr/lib/system/libmacho.dylib 0x9ac13000 - 0x9ac14fff /usr/lib/system/libsystem_sandbox.dylib 0x9bb98000 - 0x9bbc7fff /usr/lib/system/libsystem_info.dylib 0x9bf9b000 - 0x9bf9bfff /usr/lib/system/libdnsinfo.dylib 0x9bfa0000 - 0x9bfbefff /usr/lib/system/libsystem_kernel.dylib 0x9c217000 - 0x9c21ffff /usr/lib/system/libunwind.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=79248, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sun Nov 16 23:48:42 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90213000 - 0x90214fff /usr/lib/system/libsystem_blocks.dylib 0x90a16000 - 0x90a19fff /usr/lib/system/libmathCommon.A.dylib 0x90bdc000 - 0x90ca7fff /usr/lib/system/libsystem_c.dylib 0x91c77000 - 0x91c85fff /usr/lib/system/libdispatch.dylib 0x91d8b000 - 0x91d8ffff /usr/lib/system/libsystem_network.dylib 0x91d90000 - 0x91d90fff /usr/lib/system/libkeymgr.dylib 0x91dd2000 - 0x91de0fff /usr/lib/libz.1.dylib 0x92d76000 - 0x92d77fff /usr/lib/system/libunc.dylib 0x930af000 - 0x930b7fff /usr/lib/system/libcopyfile.dylib 0x9374a000 - 0x93751fff /usr/lib/system/libsystem_dnssd.dylib 0x937ef000 - 0x937f0fff /usr/lib/system/libquarantine.dylib 0x93c3a000 - 0x93c42fff /usr/lib/system/liblaunch.dylib 0x93c4f000 - 0x93c56fff /usr/lib/system/libsystem_notify.dylib 0x93c64000 - 0x93c68fff /usr/lib/system/libcache.dylib 0x93dfa000 - 0x93e10fff /usr/lib/system/libxpc.dylib 0x94dcc000 - 0x94dfafff /usr/lib/libSystem.B.dylib 0x94dfb000 - 0x94e3efff /usr/lib/system/libcommonCrypto.dylib 0x95b60000 - 0x95b61fff /usr/lib/system/libremovefile.dylib 0x95d3a000 - 0x95d3dfff /usr/lib/system/libcompiler_rt.dylib 0x962c4000 - 0x962c6fff /usr/lib/system/libdyld.dylib 0x99cc4000 - 0x99cc9fff /usr/lib/system/libmacho.dylib 0x9ac13000 - 0x9ac14fff /usr/lib/system/libsystem_sandbox.dylib 0x9bb98000 - 0x9bbc7fff /usr/lib/system/libsystem_info.dylib 0x9bf9b000 - 0x9bf9bfff /usr/lib/system/libdnsinfo.dylib 0x9bfa0000 - 0x9bfbefff /usr/lib/system/libsystem_kernel.dylib 0x9c217000 - 0x9c21ffff /usr/lib/system/libunwind.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=79248, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Nov 2014 20:24:16	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	311,040	676,386	2.1746
15 Nov 2014 19:05:25	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	285,120	621,164	2.1786
14 Nov 2014 04:22:28	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	259,200	565,906	2.1833
12 Nov 2014 06:48:23	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	233,280	510,448	2.1881
11 Nov 2014 12:18:44	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	207,360	455,617	2.1972
10 Nov 2014 13:36:22	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	181,440	399,024	2.1992
08 Nov 2014 17:14:56	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	155,520	339,796	2.1849
07 Nov 2014 17:40:53	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	129,600	283,252	2.1856
05 Nov 2014 09:10:55	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	103,680	225,938	2.1792
03 Nov 2014 15:34:27	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	77,760	169,561	2.1806
02 Nov 2014 20:13:24	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	51,840	113,007	2.1799
01 Nov 2014 19:52:58	1319483	17324175	hadcm3n_s19g_1940_40_009094797_1	25,920	56,596	2.1835