Task 16102402

Name	hadcm3n_7l9m_1980_40_008438845_3
Workunit	8589701
Created	3 Dec 2013, 0:04:45 UTC
Sent	3 Dec 2013, 0:05:03 UTC
Report deadline	4 Mar 2014, 7:32:14 UTC
Received	5 Jan 2014, 6:43:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1197259
Run time	27 days 22 hours 13 min 4 sec
CPU time	17 days 1 hours 26 min 58 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	3.20 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> 21:07:51 (56710): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:25:57 (11512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:20:07 (48181): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:20:08 (48181): No heartbeat from core client for 30 sec - exiting 00:20:09 (48181): No heartbeat from core client for 30 sec - exiting 00:20:10 (48181): No heartbeat from core client for 30 sec - exiting 00:20:11 (48181): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:44:35 (33513): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:21:58 (90691): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:26:16 (11480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:28:23 (18721): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:28:24 (18721): No heartbeat from core client for 30 sec - exiting 00:29:36 (21627): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:29:42 (21627): No heartbeat from core client for 30 sec - exiting 00:29:43 (21627): No heartbeat from core client for 30 sec - exiting 00:29:44 (21627): No heartbeat from core client for 30 sec - exiting 00:29:45 (21627): No heartbeat from core client for 30 sec - exiting 00:29:46 (21627): No heartbeat from core client for 30 sec - exiting 00:29:47 (21627): No heartbeat from core client for 30 sec - exiting 00:29:48 (21627): No heartbeat from core client for 30 sec - exiting 00:29:49 (21627): No heartbeat from core client for 30 sec - exiting 00:29:50 (21627): No heartbeat from core client for 30 sec - exiting 00:29:51 (21627): No heartbeat from core client for 30 sec - exiting 00:29:52 (21627): No heartbeat from core client for 30 sec - exiting 00:29:53 (21627): No heartbeat from core client for 30 sec - exiting 01:31:07 (23510): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:18:22 (24958): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:18:23 (24958): No heartbeat from core client for 30 sec - exiting 01:18:24 (24958): No heartbeat from core client for 30 sec - exiting 01:18:25 (24958): No heartbeat from core client for 30 sec - exiting 00:06:14 (87043): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:00 (29997): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:50:39 (30982): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:17:31 (98421): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:17:32 (98421): No heartbeat from core client for 30 sec - exiting 19:17:33 (98421): No heartbeat from core client for 30 sec - exiting 01:22:53 (57221): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 00:47:16 (96022): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:17 (96022): No heartbeat from core client for 30 sec - exiting 00:47:18 (96022): No heartbeat from core client for 30 sec - exiting 00:47:19 (96022): No heartbeat from core client for 30 sec - exiting 00:47:20 (96022): No heartbeat from core client for 30 sec - exiting 00:47:21 (96022): No heartbeat from core client for 30 sec - exiting 00:47:22 (96022): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 15:45:21 (55580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:45:23 (55580): No heartbeat from core client for 30 sec - exiting 20:46:00 (61853): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:28:19 (68763): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:28:21 (68763): No heartbeat from core client for 30 sec - exiting 01:28:22 (68763): No heartbeat from core client for 30 sec - exiting 01:28:23 (68763): No heartbeat from core client for 30 sec - exiting 01:28:24 (68763): No heartbeat from core client for 30 sec - exiting 01:28:25 (68763): No heartbeat from core client for 30 sec - exiting 01:28:26 (68763): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 00:34:31 (96352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:35:30 (58882): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:53:51 (69002): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:53:37 (17739): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:53:39 (17739): No heartbeat from core client for 30 sec - exiting 18:32:24 (21886): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:37:34 (28831): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:37:35 (28831): No heartbeat from core client for 30 sec - exiting 00:37:36 (28831): No heartbeat from core client for 30 sec - exiting 00:37:37 (28831): No heartbeat from core client for 30 sec - exiting 00:37:38 (28831): No heartbeat from core client for 30 sec - exiting 00:37:39 (28831): No heartbeat from core client for 30 sec - exiting 00:37:40 (28831): No heartbeat from core client for 30 sec - exiting 00:37:41 (28831): No heartbeat from core client for 30 sec - exiting 00:37:42 (28831): No heartbeat from core client for 30 sec - exiting 00:37:43 (28831): No heartbeat from core client for 30 sec - exiting 00:37:44 (28831): No heartbeat from core client for 30 sec - exiting 00:37:45 (28831): No heartbeat from core client for 30 sec - exiting 00:54:53 (37361): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:59:58 (76007): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:01 (76007): No heartbeat from core client for 30 sec - exiting 01:00:02 (76007): No heartbeat from core client for 30 sec - exiting 01:00:03 (76007): No heartbeat from core client for 30 sec - exiting 01:00:04 (76007): No heartbeat from core client for 30 sec - exiting 01:00:05 (76007): No heartbeat from core client for 30 sec - exiting 01:00:06 (76007): No heartbeat from core client for 30 sec - exiting 01:00:07 (76007): No heartbeat from core client for 30 sec - exiting 01:11:51 (96392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:27:12 (98349): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:27:13 (98349): No heartbeat from core client for 30 sec - exiting 03:40:14 (21409): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:08:47 (89975): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Jan 4 23:08:57 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcc78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92a03000 - 0x92a04fff /usr/lib/system/libsystem_blocks.dylib 0x92d55000 - 0x92d56fff /usr/lib/system/libunc.dylib 0x9378f000 - 0x9385afff /usr/lib/system/libsystem_c.dylib 0x938b4000 - 0x938b6fff /usr/lib/system/libdyld.dylib 0x9404f000 - 0x9407dfff /usr/lib/libSystem.B.dylib 0x94e48000 - 0x94e4ffff /usr/lib/system/libsystem_notify.dylib 0x94f41000 - 0x94f46fff /usr/lib/system/libmacho.dylib 0x954d5000 - 0x954ddfff /usr/lib/system/libunwind.dylib 0x96066000 - 0x96067fff /usr/lib/system/libremovefile.dylib 0x97092000 - 0x970a0fff /usr/lib/libz.1.dylib 0x975c4000 - 0x975f3fff /usr/lib/system/libsystem_info.dylib 0x977df000 - 0x977e3fff /usr/lib/system/libsystem_network.dylib 0x977e4000 - 0x977e7fff /usr/lib/system/libmathCommon.A.dylib 0x977e8000 - 0x97806fff /usr/lib/system/libsystem_kernel.dylib 0x9780c000 - 0x9780cfff /usr/lib/system/libkeymgr.dylib 0x9780d000 - 0x9781bfff /usr/lib/system/libdispatch.dylib 0x97b64000 - 0x97b65fff /usr/lib/system/libsystem_sandbox.dylib 0x98d04000 - 0x98d0cfff /usr/lib/system/liblaunch.dylib 0x99639000 - 0x99640fff /usr/lib/system/libsystem_dnssd.dylib 0x9a11b000 - 0x9a11ffff /usr/lib/system/libcache.dylib 0x9a578000 - 0x9a58efff /usr/lib/system/libxpc.dylib 0x9a58f000 - 0x9a597fff /usr/lib/system/libcopyfile.dylib 0x9a695000 - 0x9a696fff /usr/lib/system/libquarantine.dylib 0x9b698000 - 0x9b698fff /usr/lib/system/libdnsinfo.dylib 0x9b715000 - 0x9b758fff /usr/lib/system/libcommonCrypto.dylib 0x9b928000 - 0x9b92bfff /usr/lib/system/libcompiler_rt.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97842, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Jan 4 23:09:00 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcc78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92a03000 - 0x92a04fff /usr/lib/system/libsystem_blocks.dylib 0x92d55000 - 0x92d56fff /usr/lib/system/libunc.dylib 0x9378f000 - 0x9385afff /usr/lib/system/libsystem_c.dylib 0x938b4000 - 0x938b6fff /usr/lib/system/libdyld.dylib 0x9404f000 - 0x9407dfff /usr/lib/libSystem.B.dylib 0x94e48000 - 0x94e4ffff /usr/lib/system/libsystem_notify.dylib 0x94f41000 - 0x94f46fff /usr/lib/system/libmacho.dylib 0x954d5000 - 0x954ddfff /usr/lib/system/libunwind.dylib 0x96066000 - 0x96067fff /usr/lib/system/libremovefile.dylib 0x97092000 - 0x970a0fff /usr/lib/libz.1.dylib 0x975c4000 - 0x975f3fff /usr/lib/system/libsystem_info.dylib 0x977df000 - 0x977e3fff /usr/lib/system/libsystem_network.dylib 0x977e4000 - 0x977e7fff /usr/lib/system/libmathCommon.A.dylib 0x977e8000 - 0x97806fff /usr/lib/system/libsystem_kernel.dylib 0x9780c000 - 0x9780cfff /usr/lib/system/libkeymgr.dylib 0x9780d000 - 0x9781bfff /usr/lib/system/libdispatch.dylib 0x97b64000 - 0x97b65fff /usr/lib/system/libsystem_sandbox.dylib 0x98d04000 - 0x98d0cfff /usr/lib/system/liblaunch.dylib 0x99639000 - 0x99640fff /usr/lib/system/libsystem_dnssd.dylib 0x9a11b000 - 0x9a11ffff /usr/lib/system/libcache.dylib 0x9a578000 - 0x9a58efff /usr/lib/system/libxpc.dylib 0x9a58f000 - 0x9a597fff /usr/lib/system/libcopyfile.dylib 0x9a695000 - 0x9a696fff /usr/lib/system/libquarantine.dylib 0x9b698000 - 0x9b698fff /usr/lib/system/libdnsinfo.dylib 0x9b715000 - 0x9b758fff /usr/lib/system/libcommonCrypto.dylib 0x9b928000 - 0x9b92bfff /usr/lib/system/libcompiler_rt.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97842, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Jan 4 23:09:03 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcc78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92a03000 - 0x92a04fff /usr/lib/system/libsystem_blocks.dylib 0x92d55000 - 0x92d56fff /usr/lib/system/libunc.dylib 0x9378f000 - 0x9385afff /usr/lib/system/libsystem_c.dylib 0x938b4000 - 0x938b6fff /usr/lib/system/libdyld.dylib 0x9404f000 - 0x9407dfff /usr/lib/libSystem.B.dylib 0x94e48000 - 0x94e4ffff /usr/lib/system/libsystem_notify.dylib 0x94f41000 - 0x94f46fff /usr/lib/system/libmacho.dylib 0x954d5000 - 0x954ddfff /usr/lib/system/libunwind.dylib 0x96066000 - 0x96067fff /usr/lib/system/libremovefile.dylib 0x97092000 - 0x970a0fff /usr/lib/libz.1.dylib 0x975c4000 - 0x975f3fff /usr/lib/system/libsystem_info.dylib 0x977df000 - 0x977e3fff /usr/lib/system/libsystem_network.dylib 0x977e4000 - 0x977e7fff /usr/lib/system/libmathCommon.A.dylib 0x977e8000 - 0x97806fff /usr/lib/system/libsystem_kernel.dylib 0x9780c000 - 0x9780cfff /usr/lib/system/libkeymgr.dylib 0x9780d000 - 0x9781bfff /usr/lib/system/libdispatch.dylib 0x97b64000 - 0x97b65fff /usr/lib/system/libsystem_sandbox.dylib 0x98d04000 - 0x98d0cfff /usr/lib/system/liblaunch.dylib 0x99639000 - 0x99640fff /usr/lib/system/libsystem_dnssd.dylib 0x9a11b000 - 0x9a11ffff /usr/lib/system/libcache.dylib 0x9a578000 - 0x9a58efff /usr/lib/system/libxpc.dylib 0x9a58f000 - 0x9a597fff /usr/lib/system/libcopyfile.dylib 0x9a695000 - 0x9a696fff /usr/lib/system/libquarantine.dylib 0x9b698000 - 0x9b698fff /usr/lib/system/libdnsinfo.dylib 0x9b715000 - 0x9b758fff /usr/lib/system/libcommonCrypto.dylib 0x9b928000 - 0x9b92bfff /usr/lib/system/libcompiler_rt.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97842, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Jan 4 23:09:05 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcc78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92a03000 - 0x92a04fff /usr/lib/system/libsystem_blocks.dylib 0x92d55000 - 0x92d56fff /usr/lib/system/libunc.dylib 0x9378f000 - 0x9385afff /usr/lib/system/libsystem_c.dylib 0x938b4000 - 0x938b6fff /usr/lib/system/libdyld.dylib 0x9404f000 - 0x9407dfff /usr/lib/libSystem.B.dylib 0x94e48000 - 0x94e4ffff /usr/lib/system/libsystem_notify.dylib 0x94f41000 - 0x94f46fff /usr/lib/system/libmacho.dylib 0x954d5000 - 0x954ddfff /usr/lib/system/libunwind.dylib 0x96066000 - 0x96067fff /usr/lib/system/libremovefile.dylib 0x97092000 - 0x970a0fff /usr/lib/libz.1.dylib 0x975c4000 - 0x975f3fff /usr/lib/system/libsystem_info.dylib 0x977df000 - 0x977e3fff /usr/lib/system/libsystem_network.dylib 0x977e4000 - 0x977e7fff /usr/lib/system/libmathCommon.A.dylib 0x977e8000 - 0x97806fff /usr/lib/system/libsystem_kernel.dylib 0x9780c000 - 0x9780cfff /usr/lib/system/libkeymgr.dylib 0x9780d000 - 0x9781bfff /usr/lib/system/libdispatch.dylib 0x97b64000 - 0x97b65fff /usr/lib/system/libsystem_sandbox.dylib 0x98d04000 - 0x98d0cfff /usr/lib/system/liblaunch.dylib 0x99639000 - 0x99640fff /usr/lib/system/libsystem_dnssd.dylib 0x9a11b000 - 0x9a11ffff /usr/lib/system/libcache.dylib 0x9a578000 - 0x9a58efff /usr/lib/system/libxpc.dylib 0x9a58f000 - 0x9a597fff /usr/lib/system/libcopyfile.dylib 0x9a695000 - 0x9a696fff /usr/lib/system/libquarantine.dylib 0x9b698000 - 0x9b698fff /usr/lib/system/libdnsinfo.dylib 0x9b715000 - 0x9b758fff /usr/lib/system/libcommonCrypto.dylib 0x9b928000 - 0x9b92bfff /usr/lib/system/libcompiler_rt.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97842, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Jan 4 23:09:08 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcc78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92a03000 - 0x92a04fff /usr/lib/system/libsystem_blocks.dylib 0x92d55000 - 0x92d56fff /usr/lib/system/libunc.dylib 0x9378f000 - 0x9385afff /usr/lib/system/libsystem_c.dylib 0x938b4000 - 0x938b6fff /usr/lib/system/libdyld.dylib 0x9404f000 - 0x9407dfff /usr/lib/libSystem.B.dylib 0x94e48000 - 0x94e4ffff /usr/lib/system/libsystem_notify.dylib 0x94f41000 - 0x94f46fff /usr/lib/system/libmacho.dylib 0x954d5000 - 0x954ddfff /usr/lib/system/libunwind.dylib 0x96066000 - 0x96067fff /usr/lib/system/libremovefile.dylib 0x97092000 - 0x970a0fff /usr/lib/libz.1.dylib 0x975c4000 - 0x975f3fff /usr/lib/system/libsystem_info.dylib 0x977df000 - 0x977e3fff /usr/lib/system/libsystem_network.dylib 0x977e4000 - 0x977e7fff /usr/lib/system/libmathCommon.A.dylib 0x977e8000 - 0x97806fff /usr/lib/system/libsystem_kernel.dylib 0x9780c000 - 0x9780cfff /usr/lib/system/libkeymgr.dylib 0x9780d000 - 0x9781bfff /usr/lib/system/libdispatch.dylib 0x97b64000 - 0x97b65fff /usr/lib/system/libsystem_sandbox.dylib 0x98d04000 - 0x98d0cfff /usr/lib/system/liblaunch.dylib 0x99639000 - 0x99640fff /usr/lib/system/libsystem_dnssd.dylib 0x9a11b000 - 0x9a11ffff /usr/lib/system/libcache.dylib 0x9a578000 - 0x9a58efff /usr/lib/system/libxpc.dylib 0x9a58f000 - 0x9a597fff /usr/lib/system/libcopyfile.dylib 0x9a695000 - 0x9a696fff /usr/lib/system/libquarantine.dylib 0x9b698000 - 0x9b698fff /usr/lib/system/libdnsinfo.dylib 0x9b715000 - 0x9b758fff /usr/lib/system/libcommonCrypto.dylib 0x9b928000 - 0x9b92bfff /usr/lib/system/libcompiler_rt.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97842, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Sat Jan 4 23:09:11 2014 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffcc78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x92a03000 - 0x92a04fff /usr/lib/system/libsystem_blocks.dylib 0x92d55000 - 0x92d56fff /usr/lib/system/libunc.dylib 0x9378f000 - 0x9385afff /usr/lib/system/libsystem_c.dylib 0x938b4000 - 0x938b6fff /usr/lib/system/libdyld.dylib 0x9404f000 - 0x9407dfff /usr/lib/libSystem.B.dylib 0x94e48000 - 0x94e4ffff /usr/lib/system/libsystem_notify.dylib 0x94f41000 - 0x94f46fff /usr/lib/system/libmacho.dylib 0x954d5000 - 0x954ddfff /usr/lib/system/libunwind.dylib 0x96066000 - 0x96067fff /usr/lib/system/libremovefile.dylib 0x97092000 - 0x970a0fff /usr/lib/libz.1.dylib 0x975c4000 - 0x975f3fff /usr/lib/system/libsystem_info.dylib 0x977df000 - 0x977e3fff /usr/lib/system/libsystem_network.dylib 0x977e4000 - 0x977e7fff /usr/lib/system/libmathCommon.A.dylib 0x977e8000 - 0x97806fff /usr/lib/system/libsystem_kernel.dylib 0x9780c000 - 0x9780cfff /usr/lib/system/libkeymgr.dylib 0x9780d000 - 0x9781bfff /usr/lib/system/libdispatch.dylib 0x97b64000 - 0x97b65fff /usr/lib/system/libsystem_sandbox.dylib 0x98d04000 - 0x98d0cfff /usr/lib/system/liblaunch.dylib 0x99639000 - 0x99640fff /usr/lib/system/libsystem_dnssd.dylib 0x9a11b000 - 0x9a11ffff /usr/lib/system/libcache.dylib 0x9a578000 - 0x9a58efff /usr/lib/system/libxpc.dylib 0x9a58f000 - 0x9a597fff /usr/lib/system/libcopyfile.dylib 0x9a695000 - 0x9a696fff /usr/lib/system/libquarantine.dylib 0x9b698000 - 0x9b698fff /usr/lib/system/libdnsinfo.dylib 0x9b715000 - 0x9b758fff /usr/lib/system/libcommonCrypto.dylib 0x9b928000 - 0x9b92bfff /usr/lib/system/libcompiler_rt.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=97842, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Jan 2014 03:23:35	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	829,440	1,435,405	1.7306
03 Jan 2014 07:30:23	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	803,520	1,395,982	1.7373
02 Jan 2014 13:06:44	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	777,600	1,355,867	1.7437
01 Jan 2014 16:34:39	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	751,680	1,313,991	1.7481
31 Dec 2013 20:21:13	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	725,760	1,271,526	1.7520
31 Dec 2013 00:08:01	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	699,840	1,228,304	1.7551
30 Dec 2013 01:50:14	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	673,920	1,183,635	1.7563
29 Dec 2013 03:36:25	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	648,000	1,138,329	1.7567
28 Dec 2013 02:44:06	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	622,080	1,092,627	1.7564
27 Dec 2013 03:25:29	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	596,160	1,045,838	1.7543
26 Dec 2013 02:57:26	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	570,240	998,949	1.7518
25 Dec 2013 00:28:37	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	544,320	951,647	1.7483
24 Dec 2013 00:16:31	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	518,400	904,937	1.7456
22 Dec 2013 23:13:37	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	492,480	857,771	1.7417
21 Dec 2013 23:17:09	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	466,560	811,422	1.7392
20 Dec 2013 23:29:09	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	440,640	766,389	1.7393
20 Dec 2013 01:18:40	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	414,720	721,235	1.7391
19 Dec 2013 02:53:38	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	388,800	674,493	1.7348
18 Dec 2013 04:45:46	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	362,880	629,206	1.7339
17 Dec 2013 08:36:27	1197259	16102402	hadcm3n_7l9m_1980_40_008438845_3	336,960	583,932	1.7329