Task 13540560

Name	hadcm3n_yfa6_1900_40_007517441_0
Workunit	7714916
Created	28 Oct 2011, 12:52:23 UTC
Sent	21 Nov 2011, 22:27:48 UTC
Report deadline	21 Feb 2012, 5:54:59 UTC
Received	15 Dec 2011, 1:05:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1180859
Run time	4 days 5 hours 18 min 35 sec
CPU time	3 days 23 hours 34 min 59 sec
Validate state	Invalid
Credit	3,421.44
Device peak FLOPS	3.49 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.35</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(16516,0xa0c99540) malloc: * error for object 0x80ea00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Wed Dec 7 22:18:40 2011 Thread 0 Crashed: Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 20:05:20 2011 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x001265f9 _gw_satn + 697 1 hadcm3n_um_6.07_i686-apple-darwin 0x0012b06b _gw_vert + 4875 2 hadcm3n_um_6.07_i686-apple-darwin 0x001226db _g_wave + 2907 3 hadcm3n_um_6.07_i686-apple-darwin 0x0012bc8a _gwav_intctl + 1162 4 hadcm3n_um_6.07_i686-apple-darwin 0x00124592 _gwav_ctl + 4306 5 hadcm3n_um_6.07_i686-apple-darwin 0x000447e3 _atm_phys + 24099 6 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 7 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 8 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 9 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 10 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 11 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a488c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a510429 _nanosleep + 345 2 libSystem.B.dylib 0x9a5102ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a4b6259 __pthread_start + 345 6 libSystem.B.dylib 0x9a4b60de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4348 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93ab4000 - 0x93ac2fff /usr/lib/libz.1.dylib 0x947a5000 - 0x947a8fff /usr/lib/system/libmathCommon.A.dylib 0x9a488000 - 0x9a62ffff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37747, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 20:05:26 2011 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x001265f9 _gw_satn + 697 1 hadcm3n_um_6.07_i686-apple-darwin 0x0012b06b _gw_vert + 4875 2 hadcm3n_um_6.07_i686-apple-darwin 0x001226db _g_wave + 2907 3 hadcm3n_um_6.07_i686-apple-darwin 0x0012bc8a _gwav_intctl + 1162 4 hadcm3n_um_6.07_i686-apple-darwin 0x00124592 _gwav_ctl + 4306 5 hadcm3n_um_6.07_i686-apple-darwin 0x000447e3 _atm_phys + 24099 6 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 7 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 8 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 9 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 10 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 11 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a488c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a510429 _nanosleep + 345 2 libSystem.B.dylib 0x9a5102ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a4b6259 __pthread_start + 345 6 libSystem.B.dylib 0x9a4b60de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4348 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93ab4000 - 0x93ac2fff /usr/lib/libz.1.dylib 0x947a5000 - 0x947a8fff /usr/lib/system/libmathCommon.A.dylib 0x9a488000 - 0x9a62ffff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37747, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 20:05:32 2011 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x001265f9 _gw_satn + 697 1 hadcm3n_um_6.07_i686-apple-darwin 0x0012b06b _gw_vert + 4875 2 hadcm3n_um_6.07_i686-apple-darwin 0x001226db _g_wave + 2907 3 hadcm3n_um_6.07_i686-apple-darwin 0x0012bc8a _gwav_intctl + 1162 4 hadcm3n_um_6.07_i686-apple-darwin 0x00124592 _gwav_ctl + 4306 5 hadcm3n_um_6.07_i686-apple-darwin 0x000447e3 _atm_phys + 24099 6 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 7 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 8 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 9 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 10 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 11 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a488c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a510429 _nanosleep + 345 2 libSystem.B.dylib 0x9a5102ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a4b6259 __pthread_start + 345 6 libSystem.B.dylib 0x9a4b60de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4348 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93ab4000 - 0x93ac2fff /usr/lib/libz.1.dylib 0x947a5000 - 0x947a8fff /usr/lib/system/libmathCommon.A.dylib 0x9a488000 - 0x9a62ffff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37747, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 20:05:38 2011 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x001265f9 _gw_satn + 697 1 hadcm3n_um_6.07_i686-apple-darwin 0x0012b06b _gw_vert + 4875 2 hadcm3n_um_6.07_i686-apple-darwin 0x001226db _g_wave + 2907 3 hadcm3n_um_6.07_i686-apple-darwin 0x0012bc8a _gwav_intctl + 1162 4 hadcm3n_um_6.07_i686-apple-darwin 0x00124592 _gwav_ctl + 4306 5 hadcm3n_um_6.07_i686-apple-darwin 0x000447e3 _atm_phys + 24099 6 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 7 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 8 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 9 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 10 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 11 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a488c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a510429 _nanosleep + 345 2 libSystem.B.dylib 0x9a5102ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a4b6259 __pthread_start + 345 6 libSystem.B.dylib 0x9a4b60de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4348 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93ab4000 - 0x93ac2fff /usr/lib/libz.1.dylib 0x947a5000 - 0x947a8fff /usr/lib/system/libmathCommon.A.dylib 0x9a488000 - 0x9a62ffff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37747, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 20:05:44 2011 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x001265f9 _gw_satn + 697 1 hadcm3n_um_6.07_i686-apple-darwin 0x0012b06b _gw_vert + 4875 2 hadcm3n_um_6.07_i686-apple-darwin 0x001226db _g_wave + 2907 3 hadcm3n_um_6.07_i686-apple-darwin 0x0012bc8a _gwav_intctl + 1162 4 hadcm3n_um_6.07_i686-apple-darwin 0x00124592 _gwav_ctl + 4306 5 hadcm3n_um_6.07_i686-apple-darwin 0x000447e3 _atm_phys + 24099 6 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 7 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 8 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 9 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 10 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 11 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a488c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a510429 _nanosleep + 345 2 libSystem.B.dylib 0x9a5102ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a4b6259 __pthread_start + 345 6 libSystem.B.dylib 0x9a4b60de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4348 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93ab4000 - 0x93ac2fff /usr/lib/libz.1.dylib 0x947a5000 - 0x947a8fff /usr/lib/system/libmathCommon.A.dylib 0x9a488000 - 0x9a62ffff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37747, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Dec 13 20:05:50 2011 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x001265f9 _gw_satn + 697 1 hadcm3n_um_6.07_i686-apple-darwin 0x0012b06b _gw_vert + 4875 2 hadcm3n_um_6.07_i686-apple-darwin 0x001226db _g_wave + 2907 3 hadcm3n_um_6.07_i686-apple-darwin 0x0012bc8a _gwav_intctl + 1162 4 hadcm3n_um_6.07_i686-apple-darwin 0x00124592 _gwav_ctl + 4306 5 hadcm3n_um_6.07_i686-apple-darwin 0x000447e3 _atm_phys + 24099 6 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 7 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 8 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 9 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 10 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 11 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a488c0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a510429 _nanosleep + 345 2 libSystem.B.dylib 0x9a5102ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a4b6259 __pthread_start + 345 6 libSystem.B.dylib 0x9a4b60de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4348 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x93ab4000 - 0x93ac2fff /usr/lib/libz.1.dylib 0x947a5000 - 0x947a8fff /usr/lib/system/libmathCommon.A.dylib 0x9a488000 - 0x9a62ffff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37747, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Dec 2011 12:02:49	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	285,120	328,038	1.1505
06 Dec 2011 12:15:54	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	259,200	298,145	1.1503
05 Dec 2011 10:26:23	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	233,280	268,260	1.1499
04 Dec 2011 07:44:41	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	207,360	238,524	1.1503
03 Dec 2011 07:46:01	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	181,440	208,735	1.1504
02 Dec 2011 04:55:51	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	155,520	178,949	1.1506
01 Dec 2011 12:02:15	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	129,600	149,116	1.1506
30 Nov 2011 09:11:44	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	103,680	119,362	1.1513
29 Nov 2011 13:16:48	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	77,760	89,482	1.1507
28 Nov 2011 18:13:18	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	51,840	59,712	1.1519
22 Nov 2011 22:14:44	1180859	13540560	hadcm3n_yfa6_1900_40_007517441_0	25,920	29,865	1.1522