Task 12881219

Name	hadcm3n_p3zb_1900_40_007222743_2
Workunit	7420983
Created	11 May 2011, 21:57:37 UTC
Sent	11 May 2011, 21:58:18 UTC
Report deadline	11 Aug 2011, 5:25:29 UTC
Received	6 Jun 2011, 16:02:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	968709
Run time	13 days 19 hours 52 min
CPU time	11 days 10 hours 16 min 31 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.6.20</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:49:16 (155): No heartbeat from core client for 30 sec - exiting 13:49:17 (155): No heartbeat from core client for 30 sec - exiting 13:49:18 (155): No heartbeat from core client for 30 sec - exiting 13:49:19 (155): No heartbeat from core client for 30 sec - exiting 13:49:20 (155): No heartbeat from core client for 30 sec - exiting 13:49:21 (155): No heartbeat from core client for 30 sec - exiting 13:49:22 (155): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:42:19 (181): No heartbeat from core client for 30 sec - exiting 01:42:20 (181): No heartbeat from core client for 30 sec - exiting 01:42:21 (181): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:36:37 (185): No heartbeat from core client for 30 sec - exiting 21:36:39 (185): No heartbeat from core client for 30 sec - exiting 21:36:40 (185): No heartbeat from core client for 30 sec - exiting 21:36:41 (185): No heartbeat from core client for 30 sec - exiting 21:36:42 (185): No heartbeat from core client for 30 sec - exiting 21:36:43 (185): No heartbeat from core client for 30 sec - exiting 21:36:44 (185): No heartbeat from core client for 30 sec - exiting 21:36:45 (185): No heartbeat from core client for 30 sec - exiting 21:36:46 (185): No heartbeat from core client for 30 sec - exiting 21:36:47 (185): No heartbeat from core client for 30 sec - exiting 21:36:48 (185): No heartbeat from core client for 30 sec - exiting 21:36:49 (185): No heartbeat from core client for 30 sec - exiting 21:36:50 (185): No heartbeat from core client for 30 sec - exiting 21:36:51 (185): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:47:18 (30775): No heartbeat from core client for 30 sec - exiting 19:47:19 (30775): No heartbeat from core client for 30 sec - exiting 19:47:20 (30775): No heartbeat from core client for 30 sec - exiting 19:47:21 (30775): No heartbeat from core client for 30 sec - exiting 19:47:22 (30775): No heartbeat from core client for 30 sec - exiting 19:47:23 (30775): No heartbeat from core client for 30 sec - exiting 19:47:24 (30775): No heartbeat from core client for 30 sec - exiting 19:47:25 (30775): No heartbeat from core client for 30 sec - exiting 19:47:26 (30775): No heartbeat from core client for 30 sec - exiting 19:47:27 (30775): No heartbeat from core client for 30 sec - exiting 19:47:28 (30775): No heartbeat from core client for 30 sec - exiting 19:47:29 (30775): No heartbeat from core client for 30 sec - exiting 19:47:30 (30775): No heartbeat from core client for 30 sec - exiting 19:47:31 (30775): No heartbeat from core client for 30 sec - exiting 19:47:32 (30775): No heartbeat from core client for 30 sec - exiting 19:47:33 (30775): No heartbeat from core client for 30 sec - exiting 19:47:34 (30775): No heartbeat from core client for 30 sec - exiting 19:47:35 (30775): No heartbeat from core client for 30 sec - exiting 19:47:36 (30775): No heartbeat from core client for 30 sec - exiting 19:47:37 (30775): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:02:21 (77879): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(176,0xa076e720) malloc: * error for object 0x803000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.5.8 build 9L30 Thu Jun 2 07:59:32 2011 BOINC backtrace under OS 10.5.x only shows exported (global) symbols and may not show the final location which caused a crash. For a better backtrace, either run under OS 10.4.x or run under OS 10.6.x or later. 0 hadcm3n_6.07_i686-apple-darwin 0x00074953 version + 61229 1 hadcm3n_6.07_i686-apple-darwin 0x0006a936 version + 20240 2 libSystem.B.dylib 0x95fe32bb _sigtramp + 43 3 ??? 0xffffffff 0x0 + 4294967295 4 libSystem.B.dylib 0x95f7df26 malloc_zone_free + 2993 5 libSystem.B.dylib 0x95f7d36d free + 106 6 hadcm3n_6.07_i686-apple-darwin 0x0000ba58 _Z12annual_cyclePKSt6vectorISsSaISsEEPKcii + 3482 7 hadcm3n_6.07_i686-apple-darwin 0x0000d36b _Z12decadalMeansiPKc + 957 8 hadcm3n_6.07_i686-apple-darwin 0x000067ff _Z9doCM3Procv + 185 9 hadcm3n_6.07_i686-apple-darwin 0x0000791c _Z8mainLoopv + 410 10 hadcm3n_6.07_i686-apple-darwin 0x000087c7 _Z6workerv + 2989 11 hadcm3n_6.07_i686-apple-darwin 0x00008aa9 main + 491 12 hadcm3n_6.07_i686-apple-darwin 0x00002676 start + 54 Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x95fab8c2 ecx: 0xbff902fc edx: 0x95f77266 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff90338 esp: 0xbff902fc ss: 0x0000001f efl: 0x00000206 eip: 0x95f77266 cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/1/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90cf5000 - 0x90cf9fff /usr/lib/system/libmathCommon.A.dylib 0x91a0c000 - 0x91a13fff /usr/lib/libgcc_s.1.dylib 0x923f7000 - 0x92454fff /usr/lib/libstdc++.6.dylib 0x93708000 - 0x93716fff /usr/lib/libz.1.dylib 0x95f76000 - 0x960ddfff /usr/lib/libSystem.B.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Jun 2011 16:05:49	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	518,400	987,382	1.9047
06 Jun 2011 16:05:48	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	492,480	941,898	1.9126
06 Jun 2011 16:05:48	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	466,560	895,777	1.9200
31 May 2011 05:37:25	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	440,640	846,754	1.9216
30 May 2011 14:40:46	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	414,720	797,933	1.9240
29 May 2011 04:11:30	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	388,800	746,826	1.9208
27 May 2011 22:46:24	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	362,880	698,510	1.9249
27 May 2011 07:54:46	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	336,960	651,949	1.9348
26 May 2011 05:49:49	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	311,040	603,087	1.9389
25 May 2011 08:38:50	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	285,120	552,382	1.9374
25 May 2011 08:38:50	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	259,200	507,024	1.9561
23 May 2011 21:59:15	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	233,280	456,716	1.9578
22 May 2011 17:04:40	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	207,360	405,064	1.9534
20 May 2011 09:19:41	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	181,440	355,948	1.9618
17 May 2011 23:32:44	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	155,520	305,815	1.9664
17 May 2011 23:32:44	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	129,600	255,500	1.9715
17 May 2011 23:32:44	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	103,680	205,518	1.9822
15 May 2011 13:55:15	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	77,760	155,088	1.9944
14 May 2011 15:25:47	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	51,840	105,616	2.0373
13 May 2011 06:47:49	968709	12881219	hadcm3n_p3zb_1900_40_007222743_2	25,920	53,859	2.0779