Task 13825589

Name	hadcm3n_o7md_1980_40_007618669_1
Workunit	7796799
Created	28 Dec 2011, 22:59:34 UTC
Sent	28 Dec 2011, 22:59:43 UTC
Report deadline	29 Mar 2012, 6:26:54 UTC
Received	2 Feb 2012, 2:29:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1032461
Run time	11 days 19 hours 49 min 25 sec
CPU time	10 days 6 hours 57 min 7 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.76 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.21</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:33:25 (68088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:40:50 (25424): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:19:51 (56108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:38:55 (71056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:39:47 (71836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:20 (71844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:13:18 (73660): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:34:22 (79420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:36:34 (88386): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:57 (91407): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:52:55 (91417): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:35:03 (1037): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:14:36 (31620): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:02:26 (52845): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:03:21 (55591): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:03:22 (55591): No heartbeat from core client for 30 sec - exiting 23:04:10 (55604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:36 (55615): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:37:24 (55709): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:38:07 (66092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:07 (66092): No heartbeat from core client for 30 sec - exiting 17:38:08 (66092): No heartbeat from core client for 30 sec - exiting 17:38:09 (66092): No heartbeat from core client for 30 sec - exiting 17:38:10 (66092): No heartbeat from core client for 30 sec - exiting 17:38:11 (66092): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:39:57 (66100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:41:05 (73813): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:41:56 (73825): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:38:42 (73834): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:33:57 (79827): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:34:52 (81741): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:34:53 (81741): No heartbeat from core client for 30 sec - exiting 13:35:51 (81751): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:43 (86216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:28:09 (86384): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:45:07 (89297): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 22:45:08 (89297): No heartbeat from core client for 30 sec - exiting 11:38:56 (96235): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:39:46 (96420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:39:48 (96428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Jan 31 19:29:26 2012 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x00249c2c _rad_ctl + 8908 1 hadcm3n_um_6.07_i686-apple-darwin 0x00040abc _atm_phys + 8444 2 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 3 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 4 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a7ffc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a887429 _nanosleep + 345 2 libSystem.B.dylib 0x9a8872ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a82d259 __pthread_start + 345 6 libSystem.B.dylib 0x9a82d0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbea13628 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x94b97000 - 0x94b9afff /usr/lib/system/libmathCommon.A.dylib 0x960e2000 - 0x960f0fff /usr/lib/libz.1.dylib 0x9a7ff000 - 0x9a9a6fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2214, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Jan 31 19:29:30 2012 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x00249c2c _rad_ctl + 8908 1 hadcm3n_um_6.07_i686-apple-darwin 0x00040abc _atm_phys + 8444 2 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 3 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 4 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a7ffc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a887429 _nanosleep + 345 2 libSystem.B.dylib 0x9a8872ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a82d259 __pthread_start + 345 6 libSystem.B.dylib 0x9a82d0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbea13628 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x94b97000 - 0x94b9afff /usr/lib/system/libmathCommon.A.dylib 0x960e2000 - 0x960f0fff /usr/lib/libz.1.dylib 0x9a7ff000 - 0x9a9a6fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2214, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Jan 31 19:29:33 2012 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x00249c2c _rad_ctl + 8908 1 hadcm3n_um_6.07_i686-apple-darwin 0x00040abc _atm_phys + 8444 2 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 3 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 4 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a7ffc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a887429 _nanosleep + 345 2 libSystem.B.dylib 0x9a8872ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a82d259 __pthread_start + 345 6 libSystem.B.dylib 0x9a82d0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbea13628 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x94b97000 - 0x94b9afff /usr/lib/system/libmathCommon.A.dylib 0x960e2000 - 0x960f0fff /usr/lib/libz.1.dylib 0x9a7ff000 - 0x9a9a6fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2214, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Jan 31 19:29:37 2012 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x00249c2c _rad_ctl + 8908 1 hadcm3n_um_6.07_i686-apple-darwin 0x00040abc _atm_phys + 8444 2 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 3 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 4 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a7ffc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a887429 _nanosleep + 345 2 libSystem.B.dylib 0x9a8872ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a82d259 __pthread_start + 345 6 libSystem.B.dylib 0x9a82d0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbea13628 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x94b97000 - 0x94b9afff /usr/lib/system/libmathCommon.A.dylib 0x960e2000 - 0x960f0fff /usr/lib/libz.1.dylib 0x9a7ff000 - 0x9a9a6fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2214, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Jan 31 19:29:40 2012 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x00249c2c _rad_ctl + 8908 1 hadcm3n_um_6.07_i686-apple-darwin 0x00040abc _atm_phys + 8444 2 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 3 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 4 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a7ffc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a887429 _nanosleep + 345 2 libSystem.B.dylib 0x9a8872ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a82d259 __pthread_start + 345 6 libSystem.B.dylib 0x9a82d0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbea13628 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x94b97000 - 0x94b9afff /usr/lib/system/libmathCommon.A.dylib 0x960e2000 - 0x960f0fff /usr/lib/libz.1.dylib 0x9a7ff000 - 0x9a9a6fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2214, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.8 build 10K549 Tue Jan 31 19:29:44 2012 Thread 0 Crashed: 0 hadcm3n_um_6.07_i686-apple-darwin 0x00249c2c _rad_ctl + 8908 1 hadcm3n_um_6.07_i686-apple-darwin 0x00040abc _atm_phys + 8444 2 hadcm3n_um_6.07_i686-apple-darwin 0x00376f9d _atm_step + 34797 3 hadcm3n_um_6.07_i686-apple-darwin 0x003955ea _u_model + 45834 4 hadcm3n_um_6.07_i686-apple-darwin 0x0038a1ed _um_shell + 5613 5 hadcm3n_um_6.07_i686-apple-darwin 0x000048e1 _main + 1153 6 hadcm3n_um_6.07_i686-apple-darwin 0x000027fb __start + 209 7 hadcm3n_um_6.07_i686-apple-darwin 0x00002729 start + 41 Thread 1: 0 libSystem.B.dylib 0x9a7ffc0e _mach_wait_until + 10 1 libSystem.B.dylib 0x9a887429 _nanosleep + 345 2 libSystem.B.dylib 0x9a8872ca _usleep + 61 3 hadcm3n_um_6.07_i686-apple-darwin 0x00442ec0 __Z11boinc_sleepd + 188 4 hadcm3n_um_6.07_i686-apple-darwin 0x0043865e __Z12timer_threadPv + 78 5 libSystem.B.dylib 0x9a82d259 __pthread_start + 345 6 libSystem.B.dylib 0x9a82d0de _thread_start + 34 Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbea13628 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x94b97000 - 0x94b9afff /usr/lib/system/libmathCommon.A.dylib 0x960e2000 - 0x960f0fff /usr/lib/libz.1.dylib 0x9a7ff000 - 0x9a9a6fff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2214, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Feb 2012 01:57:40	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	777,600	915,502	1.1773
31 Jan 2012 13:32:06	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	751,680	885,509	1.1780
29 Jan 2012 15:34:55	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	725,760	855,277	1.1785
27 Jan 2012 04:13:10	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	699,840	825,325	1.1793
24 Jan 2012 10:25:09	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	673,920	794,790	1.1794
23 Jan 2012 11:35:54	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	648,000	764,683	1.1801
23 Jan 2012 02:48:56	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	622,080	734,771	1.1812
22 Jan 2012 17:23:02	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	596,160	704,296	1.1814
17 Jan 2012 09:03:12	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	570,240	673,996	1.1820
16 Jan 2012 13:56:22	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	544,320	642,549	1.1805
15 Jan 2012 18:02:42	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	518,400	611,635	1.1799
13 Jan 2012 13:03:35	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	492,480	579,610	1.1769
12 Jan 2012 14:34:58	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	466,560	547,945	1.1744
11 Jan 2012 23:15:26	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	440,640	517,110	1.1735
11 Jan 2012 13:50:20	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	414,720	486,562	1.1732
10 Jan 2012 06:37:51	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	388,800	456,780	1.1748
08 Jan 2012 22:05:32	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	362,880	426,235	1.1746
08 Jan 2012 11:45:34	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	336,960	394,685	1.1713
08 Jan 2012 00:13:07	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	311,040	363,455	1.1685
02 Jan 2012 19:40:49	1032461	13825589	hadcm3n_o7md_1980_40_007618669_1	285,120	332,488	1.1661