Task 12744829

Name	hadcm3n_o502_1900_40_007201813_1
Workunit	7400093
Created	28 Mar 2011, 14:12:09 UTC
Sent	30 Mar 2011, 2:21:08 UTC
Report deadline	29 Jun 2011, 9:48:19 UTC
Received	3 May 2011, 5:16:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1045545
Run time	15 days 17 hours 44 min 33 sec
CPU time	13 days 10 hours 29 min 10 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	0.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08b03a18 *** ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x6b961)[0x266961] /lib/i386-linux-gnu/libc.so.6(+0x6d28b)[0x26828b] /lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0x26b41d] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0x1ba441] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0x1ba49d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x211e37] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 00110000-001ef000 r-xp 00000000 08:01 151860 /usr/lib/libstdc++.so.6.0.14 001ef000-001f3000 r--p 000de000 08:01 151860 /usr/lib/libstdc++.so.6.0.14 001f3000-001f4000 rw-p 000e2000 08:01 151860 /usr/lib/libstdc++.so.6.0.14 001f4000-001fb000 rw-p 00000000 00:00 0 001fb000-00355000 r-xp 00000000 08:01 1006 /lib/i386-linux-gnu/libc-2.13.so 00355000-00356000 ---p 0015a000 08:01 1006 /lib/i386-linux-gnu/libc-2.13.so 00356000-00358000 r--p 0015a000 08:01 1006 /lib/i386-linux-gnu/libc-2.13.so 00358000-00359000 rw-p 0015c000 08:01 1006 /lib/i386-linux-gnu/libc-2.13.so 00359000-0035c000 rw-p 00000000 00:00 0 004fa000-00516000 r-xp 00000000 08:01 5596 /lib/i386-linux-gnu/ld-2.13.so 00516000-00517000 r--p 0001b000 08:01 5596 /lib/i386-linux-gnu/ld-2.13.so 00517000-00518000 rw-p 0001c000 08:01 5596 /lib/i386-linux-gnu/ld-2.13.so 00879000-0089d000 r-xp 00000000 08:01 5595 /lib/i386-linux-gnu/libm-2.13.so 0089d000-0089e000 r--p 00023000 08:01 5595 /lib/i386-linux-gnu/libm-2.13.so 0089e000-0089f000 rw-p 00024000 08:01 5595 /lib/i386-linux-gnu/libm-2.13.so 00905000-00907000 r-xp 00000000 08:01 5593 /lib/i386-linux-gnu/libdl-2.13.so 00907000-00908000 r--p 00001000 08:01 5593 /lib/i386-linux-gnu/libdl-2.13.so 00908000-00909000 rw-p 00002000 08:01 5593 /lib/i386-linux-gnu/libdl-2.13.so 00b83000-00b98000 r-xp 00000000 08:01 5591 /lib/i386-linux-gnu/libpthread-2.13.so 00b98000-00b99000 r--p 00015000 08:01 5591 /lib/i386-linux-gnu/libpthread-2.13.so 00b99000-00b9a000 rw-p 00016000 08:01 5591 /lib/i386-linux-gnu/libpthread-2.13.so 00b9a000-00b9c000 rw-p 00000000 00:00 0 00bab000-00bac000 r-xp 00000000 00:00 0 [vdso] 00ce5000-00cff000 r-xp 00000000 08:01 592 /lib/libgcc_s.so.1 00cff000-00d00000 r--p 00019000 08:01 592 /lib/libgcc_s.so.1 00d00000-00d01000 rw-p 0001a000 08:01 592 /lib/libgcc_s.so.1 08048000-080e3000 r-xp 00000000 08:01 262884 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:01 262884 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08aae000-08b13000 rw-p 00000000 00:00 0 [heap] b7200000-b7221000 rw-p 00000000 00:00 0 b7221000-b7300000 ---p 00000000 00:00 0 b73fa000-b7875000 rw-s 00000000 08:01 1312704 /var/lib/boinc-client/slots/0/128985 b7875000-b7879000 rw-p 00000000 00:00 0 b788a000-b788b000 rw-p 00000000 00:00 0 b788b000-b788c000 ---p 00000000 00:00 0 b788c000-b788f000 rw-p 00000000 00:00 0 b788f000-b7891000 rw-s 00000000 08:01 1312545 /var/lib/boinc-client/slots/0/boinc_mmap_file b7891000-b7893000 rw-p 00000000 00:00 0 bfeb4000-bff24000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xbab400] [0xbab416] /lib/i386-linux-gnu/libc.so.6(gsignal+0x51)[0x225e71] /lib/i386-linux-gnu/libc.so.6(abort+0x17e)[0x22934e] /lib/i386-linux-gnu/libc.so.6(+0x61577)[0x25c577] /lib/i386-linux-gnu/libc.so.6(+0x6b961)[0x266961] /lib/i386-linux-gnu/libc.so.6(+0x6d28b)[0x26828b] /lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0x26b41d] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0x1ba441] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0x1ba49d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x211e37] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 May 2011 05:16:21	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	259,200	1,160,943	4.4789
01 May 2011 18:40:22	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	233,280	1,046,441	4.4858
27 Apr 2011 13:27:05	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	207,360	930,200	4.4859
25 Apr 2011 20:19:23	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	181,440	813,783	4.4851
20 Apr 2011 19:35:23	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	155,520	696,748	4.4801
20 Apr 2011 19:35:23	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	129,600	579,873	4.4743
11 Apr 2011 22:16:18	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	103,680	462,608	4.4619
09 Apr 2011 08:43:52	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	77,760	346,522	4.4563
06 Apr 2011 05:12:58	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	51,840	231,578	4.4672
02 Apr 2011 14:37:11	1045545	12744829	hadcm3n_o502_1900_40_007201813_1	25,920	116,228	4.4841