Task 13612330

Name	hadcm3n_o2s8_1980_40_007540295_3
Workunit	7737527
Created	6 Nov 2011, 8:46:43 UTC
Sent	6 Nov 2011, 9:10:32 UTC
Report deadline	5 Feb 2012, 16:37:43 UTC
Received	18 Nov 2011, 9:08:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1171411
Run time	11 days 18 hours 17 min 38 sec
CPU time	9 days 3 hours 24 min 17 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.65 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (!prev): 0x09ba01d0 *** ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x6ebc2)[0xb75b3bc2] /lib/i386-linux-gnu/libc.so.6(+0x6f862)[0xb75b4862] /lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0xb75b794d] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xb77b680f] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xb77b686b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb755e113] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:02 4981530 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:02 4981530 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 09adc000-09bdc000 rw-p 00000000 00:00 0 [heap] b6f00000-b6f21000 rw-p 00000000 00:00 0 b6f21000-b7000000 ---p 00000000 00:00 0 b70c7000-b7542000 rw-s 00000000 08:02 3147409 /var/lib/boinc-client/slots/5/134460 b7542000-b7545000 rw-p 00000000 00:00 0 b7545000-b76bb000 r-xp 00000000 08:02 4326342 /lib/i386-linux-gnu/libc-2.13.so b76bb000-b76bd000 r--p 00176000 08:02 4326342 /lib/i386-linux-gnu/libc-2.13.so b76bd000-b76be000 rw-p 00178000 08:02 4326342 /lib/i386-linux-gnu/libc-2.13.so b76be000-b76c1000 rw-p 00000000 00:00 0 b76c1000-b76dd000 r-xp 00000000 08:02 4326336 /lib/i386-linux-gnu/libgcc_s.so.1 b76dd000-b76de000 r--p 0001b000 08:02 4326336 /lib/i386-linux-gnu/libgcc_s.so.1 b76de000-b76df000 rw-p 0001c000 08:02 4326336 /lib/i386-linux-gnu/libgcc_s.so.1 b76df000-b7707000 r-xp 00000000 08:02 4332043 /lib/i386-linux-gnu/libm-2.13.so b7707000-b7708000 r--p 00028000 08:02 4332043 /lib/i386-linux-gnu/libm-2.13.so b7708000-b7709000 rw-p 00029000 08:02 4332043 /lib/i386-linux-gnu/libm-2.13.so b7709000-b77e7000 r-xp 00000000 08:02 1185588 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16 b77e7000-b77e8000 ---p 000de000 08:02 1185588 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16 b77e8000-b77ec000 r--p 000de000 08:02 1185588 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16 b77ec000-b77ed000 rw-p 000e2000 08:02 1185588 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16 b77ed000-b77f5000 rw-p 00000000 00:00 0 b77f5000-b77f8000 r-xp 00000000 08:02 4332042 /lib/i386-linux-gnu/libdl-2.13.so b77f8000-b77f9000 r--p 00002000 08:02 4332042 /lib/i386-linux-gnu/libdl-2.13.so b77f9000-b77fa000 rw-p 00003000 08:02 4332042 /lib/i386-linux-gnu/libdl-2.13.so b77fa000-b7811000 r-xp 00000000 08:02 4333039 /lib/i386-linux-gnu/libpthread-2.13.so b7811000-b7812000 r--p 00016000 08:02 4333039 /lib/i386-linux-gnu/libpthread-2.13.so b7812000-b7813000 rw-p 00017000 08:02 4333039 /lib/i386-linux-gnu/libpthread-2.13.so b7813000-b7815000 rw-p 00000000 00:00 0 b7829000-b782a000 rw-p 00000000 00:00 0 b782a000-b782b000 ---p 00000000 00:00 0 b782b000-b782e000 rw-p 00000000 00:00 0 b782e000-b7830000 rw-s 00000000 08:02 3147404 /var/lib/boinc-client/slots/5/boinc_mmap_file b7830000-b7832000 rw-p 00000000 00:00 0 b7832000-b7833000 r-xp 00000000 00:00 0 [vdso] b7833000-b7851000 r-xp 00000000 08:02 4326339 /lib/i386-linux-gnu/ld-2.13.so b7851000-b7852000 r--p 0001d000 08:02 4326339 /lib/i386-linux-gnu/ld-2.13.so b7852000-b7853000 rw-p 0001e000 08:02 4326339 /lib/i386-linux-gnu/ld-2.13.so bfdf6000-bfe66000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xb7832400] [0xb7832424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb7572c8f] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb75762b5] /lib/i386-linux-gnu/libc.so.6(+0x63dfc)[0xb75a8dfc] /lib/i386-linux-gnu/libc.so.6(+0x6ebc2)[0xb75b3bc2] /lib/i386-linux-gnu/libc.so.6(+0x6f862)[0xb75b4862] /lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0xb75b794d] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xb77b680f] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xb77b686b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb755e113] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Nov 2011 09:09:18	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	259,200	789,850	3.0473
17 Nov 2011 08:48:22	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	233,280	710,815	3.0470
16 Nov 2011 06:53:55	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	207,360	632,124	3.0484
15 Nov 2011 17:07:37	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	181,440	553,751	3.0520
15 Nov 2011 17:07:37	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	155,520	474,866	3.0534
15 Nov 2011 17:07:37	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	129,600	395,571	3.0522
15 Nov 2011 17:07:37	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	103,680	316,826	3.0558
15 Nov 2011 17:07:37	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	77,760	238,232	3.0637
09 Nov 2011 06:54:14	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	51,840	156,670	3.0222
07 Nov 2011 20:54:07	1171411	13612330	hadcm3n_o2s8_1980_40_007540295_3	25,920	78,436	3.0261