Task 13624553

Name	hadcm3n_ylt1_1940_40_007538581_3
Workunit	7735813
Created	9 Nov 2011, 20:47:06 UTC
Sent	9 Nov 2011, 20:52:53 UTC
Report deadline	9 Feb 2012, 4:20:04 UTC
Received	13 Dec 2011, 20:12:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	393617
Run time	8 days 0 hours 54 min 16 sec
CPU time	6 days 21 hours 10 min 20 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	3.80 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x82d004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x848604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x848600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x282ca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x2848000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x82d004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x848604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x848600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x282ca04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x282e204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x282e200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x282e204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x282e200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x802804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x802800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x1011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x1011e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x801204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1212,0xa08f9540) malloc: * error for object 0x801200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 17:08:47 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x9604fc5a ___kill + 10 1 libSystem.B.dylib 0x960e25a5 _raise + 26 2 libSystem.B.dylib 0x960f86e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x95feec0e _mach_wait_until + 10 1 libSystem.B.dylib 0x96076429 _nanosleep + 345 2 libSystem.B.dylib 0x960762ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9601c259 __pthread_start + 345 6 libSystem.B.dylib 0x9601c0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc928 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90268000 - 0x9026bfff /usr/lib/system/libmathCommon.A.dylib 0x95f9c000 - 0x95faafff /usr/lib/libz.1.dylib 0x95fee000 - 0x96195fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10526, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 17:08:50 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x9604fc5a ___kill + 10 1 libSystem.B.dylib 0x960e25a5 _raise + 26 2 libSystem.B.dylib 0x960f86e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x95feec0e _mach_wait_until + 10 1 libSystem.B.dylib 0x96076429 _nanosleep + 345 2 libSystem.B.dylib 0x960762ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9601c259 __pthread_start + 345 6 libSystem.B.dylib 0x9601c0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc928 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90268000 - 0x9026bfff /usr/lib/system/libmathCommon.A.dylib 0x95f9c000 - 0x95faafff /usr/lib/libz.1.dylib 0x95fee000 - 0x96195fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10526, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 17:08:53 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x9604fc5a ___kill + 10 1 libSystem.B.dylib 0x960e25a5 _raise + 26 2 libSystem.B.dylib 0x960f86e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x95feec0e _mach_wait_until + 10 1 libSystem.B.dylib 0x96076429 _nanosleep + 345 2 libSystem.B.dylib 0x960762ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9601c259 __pthread_start + 345 6 libSystem.B.dylib 0x9601c0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc928 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90268000 - 0x9026bfff /usr/lib/system/libmathCommon.A.dylib 0x95f9c000 - 0x95faafff /usr/lib/libz.1.dylib 0x95fee000 - 0x96195fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10526, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 17:09:06 2011 Thread 0 Crashed: CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 17:09:26 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x9604fc5a ___kill + 10 1 libSystem.B.dylib 0x960e25a5 _raise + 26 2 libSystem.B.dylib 0x960f86e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x95feec0e _mach_wait_until + 10 1 libSystem.B.dylib 0x96076429 _nanosleep + 345 2 libSystem.B.dylib 0x960762ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9601c259 __pthread_start + 345 6 libSystem.B.dylib 0x9601c0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc928 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90268000 - 0x9026bfff /usr/lib/system/libmathCommon.A.dylib 0x95f9c000 - 0x95faafff /usr/lib/libz.1.dylib 0x95fee000 - 0x96195fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10566, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 17:09:28 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x9604fc5a ___kill + 10 1 libSystem.B.dylib 0x960e25a5 _raise + 26 2 libSystem.B.dylib 0x960f86e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x95feec0e _mach_wait_until + 10 1 libSystem.B.dylib 0x96076429 _nanosleep + 345 2 libSystem.B.dylib 0x960762ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9601c259 __pthread_start + 345 6 libSystem.B.dylib 0x9601c0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc928 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90268000 - 0x9026bfff /usr/lib/system/libmathCommon.A.dylib 0x95f9c000 - 0x95faafff /usr/lib/libz.1.dylib 0x95fee000 - 0x96195fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10566, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 17:09:31 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x9604fc5a ___kill + 10 1 libSystem.B.dylib 0x960e25a5 _raise + 26 2 libSystem.B.dylib 0x960f86e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x95feec0e _mach_wait_until + 10 1 libSystem.B.dylib 0x96076429 _nanosleep + 345 2 libSystem.B.dylib 0x960762ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9601c259 __pthread_start + 345 6 libSystem.B.dylib 0x9601c0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc928 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90268000 - 0x9026bfff /usr/lib/system/libmathCommon.A.dylib 0x95f9c000 - 0x95faafff /usr/lib/libz.1.dylib 0x95fee000 - 0x96195fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10566, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Dec 2011 20:44:56	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	414,720	587,323	1.4162
10 Dec 2011 22:08:51	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	388,800	550,667	1.4163
07 Dec 2011 18:30:52	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	362,880	513,729	1.4157
06 Dec 2011 10:15:25	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	336,960	476,737	1.4148
05 Dec 2011 20:40:17	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	311,040	439,442	1.4128
04 Dec 2011 20:17:15	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	285,120	402,726	1.4125
02 Dec 2011 19:31:45	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	259,200	365,776	1.4112
29 Nov 2011 20:22:20	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	233,280	328,710	1.4091
24 Nov 2011 15:28:24	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	207,360	292,311	1.4097
23 Nov 2011 15:23:53	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	181,440	255,856	1.4101
21 Nov 2011 17:25:50	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	155,520	219,781	1.4132
20 Nov 2011 13:28:53	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	129,600	183,287	1.4143
16 Nov 2011 22:15:32	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	103,680	146,465	1.4127
15 Nov 2011 17:58:13	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	77,760	109,970	1.4142
15 Nov 2011 17:58:13	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	51,840	73,465	1.4171
15 Nov 2011 17:58:13	393617	13624553	hadcm3n_ylt1_1940_40_007538581_3	25,920	36,742	1.4175