Task 13632124

Name	hadcm3n_006s_1940_40_007545272_0
Workunit	7742504
Created	10 Nov 2011, 9:22:48 UTC
Sent	15 Nov 2011, 17:15:45 UTC
Report deadline	15 Feb 2012, 0:42:56 UTC
Received	11 Dec 2011, 12:32:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1173806
Run time	16 days 6 hours 18 min 12 sec
CPU time	9 days 17 hours 42 min 21 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.29 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.59</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08640090 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6b511)[0xf7552511] /lib32/libc.so.6(+0x6ce1b)[0xf7553e1b] /lib32/libc.so.6(cfree+0x6d)[0xf7556f8d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7730841] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf773089d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf74fde37] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 fc:01 2231619 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 fc:01 2231619 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 085ea000-08650000 rw-p 00000000 00:00 0 [heap] f6f00000-f6f21000 rw-p 00000000 00:00 0 f6f21000-f7000000 ---p 00000000 00:00 0 f7069000-f74e4000 rw-s 00000000 fc:01 2631044 /var/lib/boinc-client/slots/4/131680 f74e4000-f74e7000 rw-p 00000000 00:00 0 f74e7000-f763d000 r-xp 00000000 fc:01 262196 /lib32/libc-2.13.so f763d000-f763e000 ---p 00156000 fc:01 262196 /lib32/libc-2.13.so f763e000-f7640000 r--p 00156000 fc:01 262196 /lib32/libc-2.13.so f7640000-f7641000 rw-p 00158000 fc:01 262196 /lib32/libc-2.13.so f7641000-f7644000 rw-p 00000000 00:00 0 f7644000-f765e000 r-xp 00000000 fc:01 2371509 /usr/lib32/libgcc_s.so.1 f765e000-f765f000 r--p 00019000 fc:01 2371509 /usr/lib32/libgcc_s.so.1 f765f000-f7660000 rw-p 0001a000 fc:01 2371509 /usr/lib32/libgcc_s.so.1 f7660000-f7684000 r-xp 00000000 fc:01 262200 /lib32/libm-2.13.so f7684000-f7685000 r--p 00023000 fc:01 262200 /lib32/libm-2.13.so f7685000-f7686000 rw-p 00024000 fc:01 262200 /lib32/libm-2.13.so f7686000-f7765000 r-xp 00000000 fc:01 2371507 /usr/lib32/libstdc++.so.6.0.14 f7765000-f7769000 r--p 000de000 fc:01 2371507 /usr/lib32/libstdc++.so.6.0.14 f7769000-f776a000 rw-p 000e2000 fc:01 2371507 /usr/lib32/libstdc++.so.6.0.14 f776a000-f7771000 rw-p 00000000 00:00 0 f7771000-f7773000 r-xp 00000000 fc:01 262199 /lib32/libdl-2.13.so f7773000-f7774000 r--p 00001000 fc:01 262199 /lib32/libdl-2.13.so f7774000-f7775000 rw-p 00002000 fc:01 262199 /lib32/libdl-2.13.so f7775000-f778a000 r-xp 00000000 fc:01 262210 /lib32/libpthread-2.13.so f778a000-f778b000 r--p 00014000 fc:01 262210 /lib32/libpthread-2.13.so f778b000-f778c000 rw-p 00015000 fc:01 262210 /lib32/libpthread-2.13.so f778c000-f778f000 rw-p 00000000 00:00 0 f77aa000-f77ab000 rw-p 00000000 00:00 0 f77ab000-f77ac000 ---p 00000000 00:00 0 f77ac000-f77af000 rw-p 00000000 00:00 0 f77af000-f77b1000 rw-s 00000000 fc:01 2631041 /var/lib/boinc-client/slots/4/boinc_mmap_file f77b1000-f77b2000 rw-p 00000000 00:00 0 f77b2000-f77b3000 r-xp 00000000 00:00 0 [vdso] f77b3000-f77cf000 r-xp 00000000 fc:01 262193 /lib32/ld-2.13.so f77cf000-f77d0000 r--p 0001b000 fc:01 262193 /lib32/ld-2.13.so f77d0000-f77d1000 rw-p 0001c000 fc:01 262193 /lib32/ld-2.13.so fff07000-fff76000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf77b2400] [0xf77b2425] /lib32/libc.so.6(gsignal+0x51)[0xf7511ea1] /lib32/libc.so.6(abort+0x17e)[0xf75152ce] /lib32/libc.so.6(+0x61277)[0xf7548277] /lib32/libc.so.6(+0x6b511)[0xf7552511] /lib32/libc.so.6(+0x6ce1b)[0xf7553e1b] /lib32/libc.so.6(cfree+0x6d)[0xf7556f8d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7730841] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf773089d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf74fde37] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Dec 2011 12:37:41	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	259,200	841,334	3.2459
06 Dec 2011 15:37:05	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	233,280	756,680	3.2437
03 Dec 2011 21:57:01	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	207,360	673,486	3.2479
30 Nov 2011 00:18:58	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	181,440	589,843	3.2509
27 Nov 2011 19:35:34	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	155,520	505,680	3.2515
25 Nov 2011 22:47:02	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	129,600	422,811	3.2624
22 Nov 2011 08:53:50	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	103,680	342,063	3.2992
20 Nov 2011 21:20:24	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	77,760	257,287	3.3087
19 Nov 2011 20:39:38	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	51,840	169,657	3.2727
18 Nov 2011 07:43:55	1173806	13632124	hadcm3n_006s_1940_40_007545272_0	25,920	84,551	3.2620