climateprediction.net home page
Task 15986273

Task 15986273

Name hadcm3n_7vvk_1980_40_008452595_0
Workunit 8603451
Created 30 Aug 2013, 16:40:19 UTC
Sent 18 Sep 2013, 23:18:22 UTC
Report deadline 19 Dec 2013, 6:45:33 UTC
Received 17 Oct 2013, 7:08:56 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 861060
Run time 10 days 23 hours 33 min 31 sec
CPU time 10 days 22 hours 23 min 58 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.64 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:55:06 (10504): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:55:56 (13903): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:38:30 (2591): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
08:25:24 (26961): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
08:11:15 (6837): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:18:18 (6879): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
07:03:18 (12432): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:13:44 (19391): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:22:57 (28918): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:25:26 (496): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:27:31 (520): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:30:53 (536): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:33:06 (552): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:33:44 (568): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:50:05 (972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:06:27 (1021): No heartbeat from core client for 30 sec - exiting
22:06:28 (1021): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
22:06:29 (1021): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09e97a88 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6efe1)[0xf74e1fe1]
/lib32/libc.so.6(+0x70868)[0xf74e3868]
/lib32/libc.so.6(cfree+0x6d)[0xf74e694d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76d3a91]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76d3aed]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7489bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:02 49242                              /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:02 49242                              /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09e48000-09ead000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6ff6000-f7471000 rw-s 00000000 08:02 703478                             /var/lib/boinc-client/slots/9/139020
f7471000-f7473000 rw-p 00000000 00:00 0 
f7473000-f75cc000 r-xp 00000000 08:01 32275                              /lib32/libc-2.11.1.so
f75cc000-f75cd000 ---p 00159000 08:01 32275                              /lib32/libc-2.11.1.so
f75cd000-f75cf000 r--p 00159000 08:01 32275                              /lib32/libc-2.11.1.so
f75cf000-f75d0000 rw-p 0015b000 08:01 32275                              /lib32/libc-2.11.1.so
f75d0000-f75d3000 rw-p 00000000 00:00 0 
f75d3000-f75f0000 r-xp 00000000 08:01 337668                             /usr/lib32/libgcc_s.so.1
f75f0000-f75f1000 r--p 0001c000 08:01 337668                             /usr/lib32/libgcc_s.so.1
f75f1000-f75f2000 rw-p 0001d000 08:01 337668                             /usr/lib32/libgcc_s.so.1
f75f2000-f7616000 r-xp 00000000 08:01 32233                              /lib32/libm-2.11.1.so
f7616000-f7617000 r--p 00023000 08:01 32233                              /lib32/libm-2.11.1.so
f7617000-f7618000 rw-p 00024000 08:01 32233                              /lib32/libm-2.11.1.so
f7618000-f7701000 r-xp 00000000 08:01 337422                             /usr/lib32/libstdc++.so.6.0.13
f7701000-f7705000 r--p 000e9000 08:01 337422                             /usr/lib32/libstdc++.so.6.0.13
f7705000-f7706000 rw-p 000ed000 08:01 337422                             /usr/lib32/libstdc++.so.6.0.13
f7706000-f770d000 rw-p 00000000 00:00 0 
f770d000-f770f000 r-xp 00000000 08:01 32271                              /lib32/libdl-2.11.1.so
f770f000-f7710000 r--p 00001000 08:01 32271                              /lib32/libdl-2.11.1.so
f7710000-f7711000 rw-p 00002000 08:01 32271                              /lib32/libdl-2.11.1.so
f7711000-f7712000 rw-p 00000000 00:00 0 
f7712000-f7727000 r-xp 00000000 08:01 32263                              /lib32/libpthread-2.11.1.so
f7727000-f7728000 r--p 00014000 08:01 32263                              /lib32/libpthread-2.11.1.so
f7728000-f7729000 rw-p 00015000 08:01 32263                              /lib32/libpthread-2.11.1.so
f7729000-f772b000 rw-p 00000000 00:00 0 
f773c000-f773d000 rw-p 00000000 00:00 0 
f773d000-f773e000 ---p 00000000 00:00 0 
f773e000-f7741000 rw-p 00000000 00:00 0 
f7741000-f7743000 rw-s 00000000 08:02 703386                             /var/lib/boinc-client/slots/9/boinc_mmap_file
f7743000-f7745000 rw-p 00000000 00:00 0 
f7745000-f7746000 r-xp 00000000 00:00 0                                  [vdso]
f7746000-f7762000 r-xp 00000000 08:01 32257                              /lib32/ld-2.11.1.so
f7762000-f7763000 r--p 0001b000 08:01 32257                              /lib32/ld-2.11.1.so
f7763000-f7764000 rw-p 0001c000 08:01 32257                              /lib32/ld-2.11.1.so
ff8a8000-ff918000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf7745400]
[0xf7745430]
/lib32/libc.so.6(gsignal+0x51)[0xf749d951]
/lib32/libc.so.6(abort+0x182)[0xf74a0d82]
/lib32/libc.so.6(+0x64eed)[0xf74d7eed]
/lib32/libc.so.6(+0x6efe1)[0xf74e1fe1]
/lib32/libc.so.6(+0x70868)[0xf74e3868]
/lib32/libc.so.6(cfree+0x6d)[0xf74e694d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76d3a91]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76d3aed]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7489bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
17 Oct 2013 01:09:33 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 518,400 944,629 1.8222
16 Oct 2013 09:24:37 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 492,480 896,518 1.8204
15 Oct 2013 17:33:44 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 466,560 850,551 1.8230
15 Oct 2013 02:12:24 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 440,640 803,556 1.8236
14 Oct 2013 11:33:23 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 414,720 755,888 1.8226
13 Oct 2013 19:20:02 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 388,800 708,449 1.8221
13 Oct 2013 03:17:25 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 362,880 659,359 1.8170
12 Oct 2013 11:38:31 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 336,960 610,864 1.8129
11 Oct 2013 20:18:35 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 311,040 563,448 1.8115
11 Oct 2013 05:02:58 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 285,120 515,000 1.8063
10 Oct 2013 13:30:36 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 259,200 468,186 1.8063
09 Oct 2013 20:54:05 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 233,280 419,731 1.7993
09 Oct 2013 04:35:54 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 207,360 373,162 1.7996
08 Oct 2013 12:23:54 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 181,440 324,914 1.7908
07 Oct 2013 19:55:46 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 155,520 275,744 1.7730
07 Oct 2013 05:38:02 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 129,600 229,705 1.7724
06 Oct 2013 14:17:56 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 103,680 181,769 1.7532
05 Oct 2013 22:14:11 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 77,760 142,143 1.8280
05 Oct 2013 06:28:07 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 51,840 95,981 1.8515
04 Oct 2013 14:21:40 861060 15986273 hadcm3n_7vvk_1980_40_008452595_0 25,920 48,934 1.8879


©2024 climateprediction.net