Task 13349196

Name	hadcm3n_o4eh_1940_40_007443549_0
Workunit	7641052
Created	9 Sep 2011, 0:07:08 UTC
Sent	9 Sep 2011, 0:09:53 UTC
Report deadline	9 Dec 2011, 7:37:04 UTC
Received	22 Sep 2011, 7:33:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1168218
Run time	10 days 6 hours 58 min 43 sec
CPU time	7 days 19 hours 31 min 45 sec
Validate state	Invalid
Credit	4,354.56
Device peak FLOPS	3.27 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.35</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(61723,0xa0ba2540) malloc: * error for object 0x4801c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:39:44 (39484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Sep 21 22:30:06 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x98c38c5a ___kill + 10 1 libSystem.B.dylib 0x98ccb5a5 _raise + 26 2 libSystem.B.dylib 0x98ce16e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x98bd7c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x98c5f429 _nanosleep + 345 2 libSystem.B.dylib 0x98c5f2ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x98c05259 __pthread_start + 345 6 libSystem.B.dylib 0x98c050de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc988 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93a1b000 - 0x93a1efff /usr/lib/system/libmathCommon.A.dylib 0x98bd7000 - 0x98d7efff /usr/lib/libSystem.B.dylib 0x9ab0a000 - 0x9ab18fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=73882, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Sep 21 22:30:08 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x98c38c5a ___kill + 10 1 libSystem.B.dylib 0x98ccb5a5 _raise + 26 2 libSystem.B.dylib 0x98ce16e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x98bd7c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x98c5f429 _nanosleep + 345 2 libSystem.B.dylib 0x98c5f2ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x98c05259 __pthread_start + 345 6 libSystem.B.dylib 0x98c050de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc988 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93a1b000 - 0x93a1efff /usr/lib/system/libmathCommon.A.dylib 0x98bd7000 - 0x98d7efff /usr/lib/libSystem.B.dylib 0x9ab0a000 - 0x9ab18fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=73882, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Sep 21 22:30:11 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x98c38c5a ___kill + 10 1 libSystem.B.dylib 0x98ccb5a5 _raise + 26 2 libSystem.B.dylib 0x98ce16e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x98bd7c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x98c5f429 _nanosleep + 345 2 libSystem.B.dylib 0x98c5f2ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x98c05259 __pthread_start + 345 6 libSystem.B.dylib 0x98c050de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc988 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93a1b000 - 0x93a1efff /usr/lib/system/libmathCommon.A.dylib 0x98bd7000 - 0x98d7efff /usr/lib/libSystem.B.dylib 0x9ab0a000 - 0x9ab18fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=73882, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Sep 21 22:30:14 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x98c38c5a ___kill + 10 1 libSystem.B.dylib 0x98ccb5a5 _raise + 26 2 libSystem.B.dylib 0x98ce16e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x98bd7c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x98c5f429 _nanosleep + 345 2 libSystem.B.dylib 0x98c5f2ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x98c05259 __pthread_start + 345 6 libSystem.B.dylib 0x98c050de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc988 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93a1b000 - 0x93a1efff /usr/lib/system/libmathCommon.A.dylib 0x98bd7000 - 0x98d7efff /usr/lib/libSystem.B.dylib 0x9ab0a000 - 0x9ab18fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=73882, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Sep 21 22:30:16 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x98c38c5a ___kill + 10 1 libSystem.B.dylib 0x98ccb5a5 _raise + 26 2 libSystem.B.dylib 0x98ce16e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x98bd7c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x98c5f429 _nanosleep + 345 2 libSystem.B.dylib 0x98c5f2ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x98c05259 __pthread_start + 345 6 libSystem.B.dylib 0x98c050de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc988 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93a1b000 - 0x93a1efff /usr/lib/system/libmathCommon.A.dylib 0x98bd7000 - 0x98d7efff /usr/lib/libSystem.B.dylib 0x9ab0a000 - 0x9ab18fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=73882, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Sep 21 22:30:19 2011 Thread 0 Crashed: 0 libSystem.B.dylib 0x98c38c5a ___kill + 10 1 libSystem.B.dylib 0x98ccb5a5 _raise + 26 2 libSystem.B.dylib 0x98ce16e4 _abort + 93 3 hadcm3n_um_6.07_i686-apple-darwin 0x00336cff _um_setup + 127 4 hadcm3n_um_6.07_i686-apple-darwin 0x00389b71 _um_shell + 3953 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x98bd7c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x98c5f429 _nanosleep + 345 2 libSystem.B.dylib 0x98c5f2ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x98c05259 __pthread_start + 345 6 libSystem.B.dylib 0x98c050de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc988 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93a1b000 - 0x93a1efff /usr/lib/system/libmathCommon.A.dylib 0x98bd7000 - 0x98d7efff /usr/lib/libSystem.B.dylib 0x9ab0a000 - 0x9ab18fff /usr/lib/libz.1.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=73882, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Sep 2011 19:38:07	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	362,880	653,305	1.8003
21 Sep 2011 00:41:30	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	336,960	608,066	1.8046
19 Sep 2011 17:48:21	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	311,040	562,551	1.8086
18 Sep 2011 21:39:01	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	285,120	517,577	1.8153
18 Sep 2011 03:11:27	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	259,200	470,962	1.8170
17 Sep 2011 09:10:28	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	233,280	424,625	1.8202
15 Sep 2011 18:01:17	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	207,360	378,417	1.8249
14 Sep 2011 21:57:09	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	181,440	330,305	1.8205
14 Sep 2011 04:34:54	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	155,520	282,493	1.8164
13 Sep 2011 10:18:09	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	129,600	234,630	1.8104
12 Sep 2011 15:35:53	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	103,680	186,631	1.8001
11 Sep 2011 20:28:48	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	77,760	139,603	1.7953
11 Sep 2011 02:11:37	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	51,840	93,348	1.8007
10 Sep 2011 07:04:26	1168218	13349196	hadcm3n_o4eh_1940_40_007443549_0	25,920	46,448	1.7920