Task 15991900

Name	hadcm3n_802h_1980_40_008458028_0
Workunit	8608884
Created	30 Aug 2013, 20:01:20 UTC
Sent	9 Sep 2013, 0:09:08 UTC
Report deadline	9 Dec 2013, 7:36:19 UTC
Received	22 Oct 2013, 3:25:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1197259
Run time	35 days 11 hours 31 min 12 sec
CPU time	20 days 5 hours 34 min 42 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.19 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 15:31:32 (87622): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:57:12 (52810): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:01:15 (61911): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:01:16 (61911): No heartbeat from core client for 30 sec - exiting 02:01:17 (61911): No heartbeat from core client for 30 sec - exiting 02:01:18 (61911): No heartbeat from core client for 30 sec - exiting 02:01:19 (61911): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 17:52:44 (43039): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:53:16 (49723): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:01:29 (1926): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:52:13 (29482): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:09:21 (31237): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:02:37 (2803): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:46:33 (47168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:36:15 (82245): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:01:35 (87418): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:03:34 (89742): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:43 (92870): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:43:08 (82632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:16:33 (16278): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:44:18 (72793): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:44:19 (72793): No heartbeat from core client for 30 sec - exiting 01:00:05 (88696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:01:17 (89110): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:36:03 (90550): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:58:55 (11578): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:13:01 (21338): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:04:27 (47395): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:10:53 (56487): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:02:40 (59612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:03:44 (94007): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:52 (96828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:53 (96828): No heartbeat from core client for 30 sec - exiting 01:59:54 (96828): No heartbeat from core client for 30 sec - exiting 01:59:55 (96828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:01:36 (23803): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:45:47 (18479): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:13:42 (84863): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:26:17 (96328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(55740,0xac36d2c0) malloc: * error for object 0x595c200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G63 Mon Oct 21 22:00:17 2013 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/6/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x92a03000 - 0x92a04fff /usr/lib/system/libsystem_blocks.dylib 0x92d55000 - 0x92d56fff /usr/lib/system/libunc.dylib 0x9378f000 - 0x9385afff /usr/lib/system/libsystem_c.dylib 0x938b4000 - 0x938b6fff /usr/lib/system/libdyld.dylib 0x9404f000 - 0x9407dfff /usr/lib/libSystem.B.dylib 0x94e48000 - 0x94e4ffff /usr/lib/system/libsystem_notify.dylib 0x94f41000 - 0x94f46fff /usr/lib/system/libmacho.dylib 0x954d5000 - 0x954ddfff /usr/lib/system/libunwind.dylib 0x96066000 - 0x96067fff /usr/lib/system/libremovefile.dylib 0x97092000 - 0x970a0fff /usr/lib/libz.1.dylib 0x975c4000 - 0x975f3fff /usr/lib/system/libsystem_info.dylib 0x977df000 - 0x977e3fff /usr/lib/system/libsystem_network.dylib 0x977e4000 - 0x977e7fff /usr/lib/system/libmathCommon.A.dylib 0x977e8000 - 0x97806fff /usr/lib/system/libsystem_kernel.dylib 0x9780c000 - 0x9780cfff /usr/lib/system/libkeymgr.dylib 0x9780d000 - 0x9781bfff /usr/lib/system/libdispatch.dylib 0x97b64000 - 0x97b65fff /usr/lib/system/libsystem_sandbox.dylib 0x97c05000 - 0x97c0efff /usr/lib/libc++abi.dylib 0x98d04000 - 0x98d0cfff /usr/lib/system/liblaunch.dylib 0x99639000 - 0x99640fff /usr/lib/system/libsystem_dnssd.dylib 0x9a11b000 - 0x9a11ffff /usr/lib/system/libcache.dylib 0x9a45d000 - 0x9a4bffff /usr/lib/libstdc++.6.dylib 0x9a578000 - 0x9a58efff /usr/lib/system/libxpc.dylib 0x9a58f000 - 0x9a597fff /usr/lib/system/libcopyfile.dylib 0x9a695000 - 0x9a696fff /usr/lib/system/libquarantine.dylib 0x9b698000 - 0x9b698fff /usr/lib/system/libdnsinfo.dylib 0x9b715000 - 0x9b758fff /usr/lib/system/libcommonCrypto.dylib 0x9b928000 - 0x9b92bfff /usr/lib/system/libcompiler_rt.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Oct 2013 03:26:08	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	1,036,800	1,748,079	1.6860
21 Oct 2013 11:34:18	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	1,010,880	1,710,962	1.6925
20 Oct 2013 17:12:16	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	984,960	1,670,143	1.6956
20 Oct 2013 00:45:31	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	959,040	1,632,180	1.7019
19 Oct 2013 06:34:47	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	933,120	1,594,669	1.7090
17 Oct 2013 16:06:49	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	907,200	1,557,395	1.7167
16 Oct 2013 23:18:26	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	881,280	1,518,951	1.7236
16 Oct 2013 01:16:25	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	855,360	1,475,525	1.7250
15 Oct 2013 02:22:29	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	829,440	1,431,257	1.7256
10 Oct 2013 04:54:04	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	803,520	1,387,155	1.7263
09 Oct 2013 06:06:18	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	777,600	1,342,499	1.7265
08 Oct 2013 07:13:13	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	751,680	1,297,148	1.7257
07 Oct 2013 08:24:58	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	725,760	1,252,230	1.7254
06 Oct 2013 11:25:48	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	699,840	1,208,508	1.7268
05 Oct 2013 14:45:04	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	673,920	1,164,845	1.7285
04 Oct 2013 17:36:21	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	648,000	1,121,732	1.7311
03 Oct 2013 16:21:05	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	622,080	1,076,311	1.7302
02 Oct 2013 15:30:31	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	596,160	1,031,219	1.7298
30 Sep 2013 23:31:33	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	570,240	986,772	1.7305
29 Sep 2013 03:06:51	1197259	15991900	hadcm3n_802h_1980_40_008458028_0	544,320	942,892	1.7322