climateprediction.net home page
Task 13443105

Task 13443105

Name hadcm3n_u23j_1980_40_007458278_4
Workunit 7655781
Created 27 Sep 2011, 3:42:37 UTC
Sent 27 Sep 2011, 3:44:06 UTC
Report deadline 27 Dec 2011, 11:11:17 UTC
Received 26 Nov 2011, 2:08:17 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1410934
Run time 32 days 18 hours 16 min 24 sec
CPU time 31 days 0 hours 31 min 16 sec
Validate state Invalid
Credit 12,441.60
Device peak FLOPS 1.86 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.34</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:46:23 (18241): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0a049530 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0x826591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0x827de8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0x82aecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xd8d741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xd8d79d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x7d1bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
00110000-00112000 r-xp 00000000 08:01 3933292    /lib/tls/i686/cmov/libdl-2.11.1.so
00112000-00113000 r--p 00001000 08:01 3933292    /lib/tls/i686/cmov/libdl-2.11.1.so
00113000-00114000 rw-p 00002000 08:01 3933292    /lib/tls/i686/cmov/libdl-2.11.1.so
006df000-006fa000 r-xp 00000000 08:01 2831       /lib/ld-2.11.1.so
006fa000-006fb000 r--p 0001a000 08:01 2831       /lib/ld-2.11.1.so
006fb000-006fc000 rw-p 0001b000 08:01 2831       /lib/ld-2.11.1.so
007bb000-0090e000 r-xp 00000000 08:01 3932600    /lib/tls/i686/cmov/libc-2.11.1.so
0090e000-0090f000 ---p 00153000 08:01 3932600    /lib/tls/i686/cmov/libc-2.11.1.so
0090f000-00911000 r--p 00153000 08:01 3932600    /lib/tls/i686/cmov/libc-2.11.1.so
00911000-00912000 rw-p 00155000 08:01 3932600    /lib/tls/i686/cmov/libc-2.11.1.so
00912000-00915000 rw-p 00000000 00:00 0 
00ab7000-00acc000 r-xp 00000000 08:01 3934177    /lib/tls/i686/cmov/libpthread-2.11.1.so
00acc000-00acd000 r--p 00014000 08:01 3934177    /lib/tls/i686/cmov/libpthread-2.11.1.so
00acd000-00ace000 rw-p 00015000 08:01 3934177    /lib/tls/i686/cmov/libpthread-2.11.1.so
00ace000-00ad0000 rw-p 00000000 00:00 0 
00be2000-00be3000 r-xp 00000000 00:00 0          [vdso]
00cd2000-00dbb000 r-xp 00000000 08:01 3933789    /usr/lib/libstdc++.so.6.0.13
00dbb000-00dbc000 ---p 000e9000 08:01 3933789    /usr/lib/libstdc++.so.6.0.13
00dbc000-00dc0000 r--p 000e9000 08:01 3933789    /usr/lib/libstdc++.so.6.0.13
00dc0000-00dc1000 rw-p 000ed000 08:01 3933789    /usr/lib/libstdc++.so.6.0.13
00dc1000-00dc8000 rw-p 00000000 00:00 0 
00ded000-00e11000 r-xp 00000000 08:01 3933694    /lib/tls/i686/cmov/libm-2.11.1.so
00e11000-00e12000 r--p 00023000 08:01 3933694    /lib/tls/i686/cmov/libm-2.11.1.so
00e12000-00e13000 rw-p 00024000 08:01 3933694    /lib/tls/i686/cmov/libm-2.11.1.so
00f93000-00fb0000 r-xp 00000000 08:01 6729       /lib/libgcc_s.so.1
00fb0000-00fb1000 r--p 0001c000 08:01 6729       /lib/libgcc_s.so.1
00fb1000-00fb2000 rw-p 0001d000 08:01 6729       /lib/libgcc_s.so.1
08048000-080e3000 r-xp 00000000 08:01 133052     /home/bob/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:01 133052     /home/bob/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09ff3000-0a05a000 rw-p 00000000 00:00 0          [heap]
b7200000-b7221000 rw-p 00000000 00:00 0 
b7221000-b7300000 ---p 00000000 00:00 0 
b7359000-b77d4000 rw-s 00000000 08:01 2883754    /home/bob/BOINC/slots/2/135450
b77d4000-b77d7000 rw-p 00000000 00:00 0 
b77e4000-b77e5000 rw-p 00000000 00:00 0 
b77e5000-b77e6000 ---p 00000000 00:00 0 
b77e6000-b77e9000 rw-p 00000000 00:00 0 
b77e9000-b77eb000 rw-s 00000000 08:01 2883751    /home/bob/BOINC/slots/2/boinc_mmap_file
b77eb000-b77ed000 rw-p 00000000 00:00 0 
bfc67000-bfcd7000 rw-p 00000000 00:00 0          [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xbe2400]
[0xbe2422]
/lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x7e5651]
/lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x7e8a82]
/lib/tls/i686/cmov/libc.so.6(+0x6149d)[0x81c49d]
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0x826591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0x827de8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0x82aecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xd8d741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xd8d79d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x7d1bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Nov 2011 21:36:23 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 1,036,800 2,680,271 2.5851
25 Nov 2011 02:39:24 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 1,010,880 2,615,007 2.5869
24 Nov 2011 07:46:20 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 984,960 2,550,031 2.5890
23 Nov 2011 13:03:12 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 959,040 2,485,055 2.5912
22 Nov 2011 16:05:43 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 933,120 2,420,672 2.5942
21 Nov 2011 15:50:20 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 907,200 2,356,351 2.5974
20 Nov 2011 16:24:33 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 881,280 2,292,407 2.6012
19 Nov 2011 15:30:31 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 855,360 2,221,426 2.5971
18 Nov 2011 16:26:14 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 829,440 2,153,050 2.5958
17 Nov 2011 14:04:45 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 803,520 2,083,653 2.5932
16 Nov 2011 09:30:18 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 777,600 2,010,505 2.5855
15 Nov 2011 16:43:09 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 751,680 1,947,430 2.5908
15 Nov 2011 16:43:11 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 725,760 1,876,608 2.5857
15 Nov 2011 16:43:10 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 699,840 1,809,174 2.5851
15 Nov 2011 16:43:12 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 673,920 1,740,437 2.5826
15 Nov 2011 16:43:12 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 648,000 1,669,943 2.5771
15 Nov 2011 16:43:10 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 622,080 1,605,049 2.5801
09 Nov 2011 18:53:42 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 596,160 1,533,993 2.5731
08 Nov 2011 13:58:37 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 570,240 1,459,791 2.5600
07 Nov 2011 14:22:15 1106339 13443105 hadcm3n_u23j_1980_40_007458278_4 544,320 1,389,672 2.5530


©2024 climateprediction.net