Task 15878932

Name	hadcm3n_zlce_1880_40_008281225_2
Workunit	8432360
Created	3 Jul 2013, 15:07:46 UTC
Sent	3 Jul 2013, 15:19:50 UTC
Report deadline	2 Oct 2013, 22:47:01 UTC
Received	18 Jul 2013, 12:56:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1291115
Run time	5 days 6 hours 7 min 41 sec
CPU time	5 days 6 hours 7 min 41 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 12:40:12 (60605): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 12:42:38 (68630): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 13:25:15 (70943): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 13:29:41 (77695): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 14:39:37 (77869): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 14:42:22 (93846): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 04:26:10 (96067): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 04:29:21 (83872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 06:54:44 (83897): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 08:02:40 (51740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 08:04:08 (88030): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 08:06:17 (88093): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:33:55 (90480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 17:08:52 (17179): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 18:44:21 (71617): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 18:45:36 (12540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 19:03:33 (12591): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:53:15 (20234): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:55:21 (45592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:08:25 (8305): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:21:11 (67683): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 20:22:29 (27419): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:28:16 (44165): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:40:15 (72537): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 12:27:48 (78486): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:36:03 (97922): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:57:03 (33445): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:53:58 (50883): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 19:56:31 (83208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 21:56:00 (85478): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 03:36:54 (23789): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... 14:14:41 (921): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:57:48 (29943): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid 11:42:16 (44608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation dyld: DYLD_ environment variables being ignored because main executable (/Volumes/big/glake/boinc/slots/10/../../switcher/switcher) is setuid or setgid hadcm3n_6.07_i686-apple-darwin(55806,0xac5cca28) malloc: * error for object 0xc281804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55806,0xac5cca28) malloc: * error for object 0xc281800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55806,0xac5cca28) malloc: * error for object 0x7a61c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55806,0xac5cca28) malloc: * error for object 0x7a61c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(55806,0xac5cca28) malloc: * error for object 0xe261c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.8.4 build 12E55 Sun Jul 14 13:31:17 2013 dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid dyld: DYLD_ environment variables being ignored because main executable (/bin/ps) is setuid or setgid Thread 0 Crashed: Thread 1: dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/atos) has __RESTRICT/__restrict section atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa78 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Volumes/big/glake/boinc/slots/10/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90008000 - 0x9001afff /usr/lib/system/libdispatch.dylib 0x908ce000 - 0x908cffff /usr/lib/system/libremovefile.dylib 0x90cf5000 - 0x90d22fff /usr/lib/system/libsystem_m.dylib 0x90d80000 - 0x90d8afff /usr/lib/system/libsystem_notify.dylib 0x90f4e000 - 0x90f5cfff /usr/lib/system/libsystem_network.dylib 0x90f5d000 - 0x90f7afff /usr/lib/system/libxpc.dylib 0x90ff3000 - 0x91015fff /usr/lib/libc++abi.dylib 0x910e9000 - 0x9114dfff /usr/lib/libstdc++.6.dylib 0x9185c000 - 0x91863fff /usr/lib/system/liblaunch.dylib 0x920e2000 - 0x920e2fff /usr/lib/system/libsystem_blocks.dylib 0x921ec000 - 0x921eefff /usr/lib/system/libdyld.dylib 0x928f1000 - 0x928f1fff /usr/lib/system/libkeymgr.dylib 0x92cf4000 - 0x92d20fff /usr/lib/system/libsystem_info.dylib 0x92d21000 - 0x92d21fff /usr/lib/libSystem.B.dylib 0x931ab000 - 0x931b2fff /usr/lib/system/libunwind.dylib 0x93639000 - 0x93685fff /usr/lib/system/libcorecrypto.dylib 0x937b5000 - 0x937b8fff /usr/lib/system/libcompiler_rt.dylib 0x93dea000 - 0x93ea7fff /usr/lib/system/libsystem_c.dylib 0x9458e000 - 0x94592fff /usr/lib/system/libcache.dylib 0x949b3000 - 0x949b4fff /usr/lib/system/libdnsinfo.dylib 0x94bcd000 - 0x94bd4fff /usr/lib/system/libsystem_dnssd.dylib 0x94bd5000 - 0x94be0fff /usr/lib/system/libcommonCrypto.dylib 0x953a2000 - 0x953a3fff /usr/lib/system/libunc.dylib 0x953d2000 - 0x953dafff /usr/lib/system/libcopyfile.dylib 0x954a7000 - 0x954a8fff /usr/lib/system/libquarantine.dylib 0x99b78000 - 0x99b7cfff /usr/lib/system/libmacho.dylib 0x99c9c000 - 0x99c9dfff /usr/lib/system/libsystem_sandbox.dylib 0x9a0df000 - 0x9a0f9fff /usr/lib/system/libsystem_kernel.dylib 0x9a190000 - 0x9a19efff /usr/lib/libz.1.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2013 16:19:08	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	259,200	454,051	1.7517
23 Jul 2013 16:19:11	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	233,280	405,074	1.7364
08 Jul 2013 15:03:57	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	207,360	359,348	1.7330
08 Jul 2013 01:10:42	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	181,440	310,214	1.7097
07 Jul 2013 06:25:36	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	155,520	263,321	1.6932
06 Jul 2013 15:24:16	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	129,600	219,611	1.6945
06 Jul 2013 05:40:56	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	103,680	174,355	1.6817
06 Jul 2013 05:02:15	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	77,760	130,080	1.6728
06 Jul 2013 04:21:24	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	51,840	87,632	1.6904
04 Jul 2013 14:27:55	1099001	15878932	hadcm3n_zlce_1880_40_008281225_2	25,920	44,010	1.6979