Task 13765554

Name	hadcm3n_t17g_1940_40_007543704_3
Workunit	7740936
Created	11 Dec 2011, 23:33:12 UTC
Sent	11 Dec 2011, 23:33:17 UTC
Report deadline	12 Mar 2012, 7:00:28 UTC
Received	4 Jan 2012, 0:58:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1184498
Run time	7 days 23 hours 17 min 25 sec
CPU time	7 days 19 hours 57 min 40 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.38 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x091e1140 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6eb72)[0xf74c4b72] /lib32/libc.so.6(+0x6f812)[0xf74c5812] /lib32/libc.so.6(cfree+0x6d)[0xf74c88cd] /usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf76c5baf] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf76c5c0b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf746f0f3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:01 7085225 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:01 7085225 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 0918b000-091f1000 rw-p 00000000 00:00 0 [heap] f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6fd8000-f7453000 rw-s 00000000 08:01 7086025 /var/lib/boinc-client/slots/2/133560 f7453000-f7456000 rw-p 00000000 00:00 0 f7456000-f75c9000 r-xp 00000000 08:01 3014666 /lib32/libc-2.13.so f75c9000-f75ca000 ---p 00173000 08:01 3014666 /lib32/libc-2.13.so f75ca000-f75cc000 r--p 00173000 08:01 3014666 /lib32/libc-2.13.so f75cc000-f75cd000 rw-p 00175000 08:01 3014666 /lib32/libc-2.13.so f75cd000-f75d0000 rw-p 00000000 00:00 0 f75d0000-f75ec000 r-xp 00000000 08:01 4464385 /usr/lib32/libgcc_s.so.1 f75ec000-f75ed000 r--p 0001b000 08:01 4464385 /usr/lib32/libgcc_s.so.1 f75ed000-f75ee000 rw-p 0001c000 08:01 4464385 /usr/lib32/libgcc_s.so.1 f75ee000-f7616000 r-xp 00000000 08:01 3014676 /lib32/libm-2.13.so f7616000-f7617000 r--p 00028000 08:01 3014676 /lib32/libm-2.13.so f7617000-f7618000 rw-p 00029000 08:01 3014676 /lib32/libm-2.13.so f7618000-f76f6000 r-xp 00000000 08:01 4465630 /usr/lib32/libstdc++.so.6.0.16 f76f6000-f76f7000 ---p 000de000 08:01 4465630 /usr/lib32/libstdc++.so.6.0.16 f76f7000-f76fb000 r--p 000de000 08:01 4465630 /usr/lib32/libstdc++.so.6.0.16 f76fb000-f76fc000 rw-p 000e2000 08:01 4465630 /usr/lib32/libstdc++.so.6.0.16 f76fc000-f7704000 rw-p 00000000 00:00 0 f7704000-f7707000 r-xp 00000000 08:01 3014667 /lib32/libdl-2.13.so f7707000-f7708000 r--p 00002000 08:01 3014667 /lib32/libdl-2.13.so f7708000-f7709000 rw-p 00003000 08:01 3014667 /lib32/libdl-2.13.so f7709000-f7720000 r-xp 00000000 08:01 3014674 /lib32/libpthread-2.13.so f7720000-f7721000 r--p 00016000 08:01 3014674 /lib32/libpthread-2.13.so f7721000-f7722000 rw-p 00017000 08:01 3014674 /lib32/libpthread-2.13.so f7722000-f7724000 rw-p 00000000 00:00 0 f773c000-f773d000 rw-p 00000000 00:00 0 f773d000-f773e000 ---p 00000000 00:00 0 f773e000-f7741000 rw-p 00000000 00:00 0 f7741000-f7743000 rw-s 00000000 08:01 7085887 /var/lib/boinc-client/slots/2/boinc_mmap_file f7743000-f7745000 rw-p 00000000 00:00 0 f7745000-f7746000 r-xp 00000000 00:00 0 [vdso] f7746000-f7764000 r-xp 00000000 08:01 5636118 /lib/i386-linux-gnu/ld-2.13.so f7764000-f7765000 r--p 0001d000 08:01 5636118 /lib/i386-linux-gnu/ld-2.13.so f7765000-f7766000 rw-p 0001e000 08:01 5636118 /lib/i386-linux-gnu/ld-2.13.so ff905000-ff976000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf7745400] [0xf7745425] /lib32/libc.so.6(gsignal+0x4f)[0xf7483c4f] /lib32/libc.so.6(abort+0x175)[0xf7487175] /lib32/libc.so.6(+0x63e8c)[0xf74b9e8c] /lib32/libc.so.6(+0x6eb72)[0xf74c4b72] /lib32/libc.so.6(+0x6f812)[0xf74c5812] /lib32/libc.so.6(cfree+0x6d)[0xf74c88cd] /usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf76c5baf] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf76c5c0b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf746f0f3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Jan 2012 00:59:10	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	259,200	676,659	2.6106
01 Jan 2012 18:40:02	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	233,280	609,149	2.6112
30 Dec 2011 17:44:16	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	207,360	540,887	2.6084
28 Dec 2011 09:14:50	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	181,440	472,652	2.6050
26 Dec 2011 07:34:54	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	155,520	404,933	2.6037
24 Dec 2011 03:23:57	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	129,600	337,552	2.6046
21 Dec 2011 17:37:14	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	103,680	270,104	2.6052
19 Dec 2011 12:13:28	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	77,760	203,137	2.6124
17 Dec 2011 03:47:41	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	51,840	135,902	2.6216
14 Dec 2011 22:16:14	1184498	13765554	hadcm3n_t17g_1940_40_007543704_3	25,920	68,253	2.6332