climateprediction.net home page
Task 13334825

Task 13334825

Name hadcm3n_o1cy_1900_40_007438994_0
Workunit 7636497
Created 5 Sep 2011, 18:14:59 UTC
Sent 6 Sep 2011, 16:34:30 UTC
Report deadline 7 Dec 2011, 0:01:41 UTC
Received 19 Nov 2011, 15:21:46 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1132398
Run time 20 days 22 hours 25 min 22 sec
CPU time 20 days 5 hours 20 min 32 sec
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 2.23 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
15:02:12 (1756): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:02:16 (1756): No heartbeat from core client for 30 sec - exiting
15:02:17 (1756): No heartbeat from core client for 30 sec - exiting
15:02:18 (1756): No heartbeat from core client for 30 sec - exiting
15:02:19 (1756): No heartbeat from core client for 30 sec - exiting
15:02:20 (1756): No heartbeat from core client for 30 sec - exiting
15:02:21 (1756): No heartbeat from core client for 30 sec - exiting
15:02:22 (1756): No heartbeat from core client for 30 sec - exiting
15:02:23 (1756): No heartbeat from core client for 30 sec - exiting
15:02:24 (1756): No heartbeat from core client for 30 sec - exiting
15:02:25 (1756): No heartbeat from core client for 30 sec - exiting
15:02:26 (1756): No heartbeat from core client for 30 sec - exiting
15:02:27 (1756): No heartbeat from core client for 30 sec - exiting
15:02:28 (1756): No heartbeat from core client for 30 sec - exiting
15:02:29 (1756): No heartbeat from core client for 30 sec - exiting
15:02:30 (1756): No heartbeat from core client for 30 sec - exiting
15:02:31 (1756): No heartbeat from core client for 30 sec - exiting
15:02:32 (1756): No heartbeat from core client for 30 sec - exiting
15:02:33 (1756): No heartbeat from core client for 30 sec - exiting
15:02:34 (1756): No heartbeat from core client for 30 sec - exiting
15:02:35 (1756): No heartbeat from core client for 30 sec - exiting
15:02:36 (1756): No heartbeat from core client for 30 sec - exiting
15:02:37 (1756): No heartbeat from core client for 30 sec - exiting
15:02:38 (1756): No heartbeat from core client for 30 sec - exiting
15:02:39 (1756): No heartbeat from core client for 30 sec - exiting
15:02:40 (1756): No heartbeat from core client for 30 sec - exiting
15:02:41 (1756): No heartbeat from core client for 30 sec - exiting
15:02:42 (1756): No heartbeat from core client for 30 sec - exiting
15:02:43 (1756): No heartbeat from core client for 30 sec - exiting
15:02:44 (1756): No heartbeat from core client for 30 sec - exiting
15:02:45 (1756): No heartbeat from core client for 30 sec - exiting
15:02:46 (1756): No heartbeat from core client for 30 sec - exiting
15:02:47 (1756): No heartbeat from core client for 30 sec - exiting
15:02:48 (1756): No heartbeat from core client for 30 sec - exiting
15:02:49 (1756): No heartbeat from core client for 30 sec - exiting
15:02:50 (1756): No heartbeat from core client for 30 sec - exiting
15:02:51 (1756): No heartbeat from core client for 30 sec - exiting
15:02:52 (1756): No heartbeat from core client for 30 sec - exiting
15:02:53 (1756): No heartbeat from core client for 30 sec - exiting
15:02:54 (1756): No heartbeat from core client for 30 sec - exiting
15:02:55 (1756): No heartbeat from core client for 30 sec - exiting
15:02:56 (1756): No heartbeat from core client for 30 sec - exiting
15:02:57 (1756): No heartbeat from core client for 30 sec - exiting
15:02:58 (1756): No heartbeat from core client for 30 sec - exiting
15:02:59 (1756): No heartbeat from core client for 30 sec - exiting
15:03:00 (1756): No heartbeat from core client for 30 sec - exiting
15:03:01 (1756): No heartbeat from core client for 30 sec - exiting
15:03:02 (1756): No heartbeat from core client for 30 sec - exiting
15:03:03 (1756): No heartbeat from core client for 30 sec - exiting
15:03:04 (1756): No heartbeat from core client for 30 sec - exiting
15:03:05 (1756): No heartbeat from core client for 30 sec - exiting
15:03:06 (1756): No heartbeat from core client for 30 sec - exiting
15:03:07 (1756): No heartbeat from core client for 30 sec - exiting
15:03:08 (1756): No heartbeat from core client for 30 sec - exiting
15:03:09 (1756): No heartbeat from core client for 30 sec - exiting
15:03:10 (1756): No heartbeat from core client for 30 sec - exiting
15:03:11 (1756): No heartbeat from core client for 30 sec - exiting
15:03:12 (1756): No heartbeat from core client for 30 sec - exiting
15:03:13 (1756): No heartbeat from core client for 30 sec - exiting
15:03:14 (1756): No heartbeat from core client for 30 sec - exiting
15:03:15 (1756): No heartbeat from core client for 30 sec - exiting
15:03:16 (1756): No heartbeat from core client for 30 sec - exiting
15:03:17 (1756): No heartbeat from core client for 30 sec - exiting
15:03:18 (1756): No heartbeat from core client for 30 sec - exiting
15:03:19 (1756): No heartbeat from core client for 30 sec - exiting
15:03:20 (1756): No heartbeat from core client for 30 sec - exiting
15:06:30 (12794): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
01:06:58 (1624): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:07:03 (1624): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:27:50 (1782): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:56:10 (1578): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:56:37 (1578): No heartbeat from core client for 30 sec - exiting
21:56:38 (1578): No heartbeat from core client for 30 sec - exiting
21:56:39 (1578): No heartbeat from core client for 30 sec - exiting
21:56:40 (1578): No heartbeat from core client for 30 sec - exiting
21:56:41 (1578): No heartbeat from core client for 30 sec - exiting
21:56:42 (1578): No heartbeat from core client for 30 sec - exiting
21:56:43 (1578): No heartbeat from core client for 30 sec - exiting
21:56:44 (1578): No heartbeat from core client for 30 sec - exiting
21:56:45 (1578): No heartbeat from core client for 30 sec - exiting
21:56:46 (1578): No heartbeat from core client for 30 sec - exiting
21:56:47 (1578): No heartbeat from core client for 30 sec - exiting
21:56:48 (1578): No heartbeat from core client for 30 sec - exiting
21:56:49 (1578): No heartbeat from core client for 30 sec - exiting
21:56:50 (1578): No heartbeat from core client for 30 sec - exiting
21:56:51 (1578): No heartbeat from core client for 30 sec - exiting
21:56:52 (1578): No heartbeat from core client for 30 sec - exiting
21:56:53 (1578): No heartbeat from core client for 30 sec - exiting
21:56:54 (1578): No heartbeat from core client for 30 sec - exiting
21:56:55 (1578): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
09:39:28 (1804): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:39:29 (1804): No heartbeat from core client for 30 sec - exiting
09:39:30 (1804): No heartbeat from core client for 30 sec - exiting
09:39:31 (1804): No heartbeat from core client for 30 sec - exiting
09:39:32 (1804): No heartbeat from core client for 30 sec - exiting
09:39:33 (1804): No heartbeat from core client for 30 sec - exiting
09:39:34 (1804): No heartbeat from core client for 30 sec - exiting
09:39:35 (1804): No heartbeat from core client for 30 sec - exiting
09:39:36 (1804): No heartbeat from core client for 30 sec - exiting
09:39:37 (1804): No heartbeat from core client for 30 sec - exiting
09:39:38 (1804): No heartbeat from core client for 30 sec - exiting
09:39:39 (1804): No heartbeat from core client for 30 sec - exiting
09:39:40 (1804): No heartbeat from core client for 30 sec - exiting
09:39:41 (1804): No heartbeat from core client for 30 sec - exiting
09:39:42 (1804): No heartbeat from core client for 30 sec - exiting
09:39:43 (1804): No heartbeat from core client for 30 sec - exiting
09:39:44 (1804): No heartbeat from core client for 30 sec - exiting
09:39:45 (1804): No heartbeat from core client for 30 sec - exiting
09:39:46 (1804): No heartbeat from core client for 30 sec - exiting
09:39:47 (1804): No heartbeat from core client for 30 sec - exiting
09:39:48 (1804): No heartbeat from core client for 30 sec - exiting
09:39:49 (1804): No heartbeat from core client for 30 sec - exiting
09:39:50 (1804): No heartbeat from core client for 30 sec - exiting
09:39:51 (1804): No heartbeat from core client for 30 sec - exiting
09:39:52 (1804): No heartbeat from core client for 30 sec - exiting
09:39:53 (1804): No heartbeat from core client for 30 sec - exiting
09:39:54 (1804): No heartbeat from core client for 30 sec - exiting
09:39:55 (1804): No heartbeat from core client for 30 sec - exiting
09:39:56 (1804): No heartbeat from core client for 30 sec - exiting
09:39:57 (1804): No heartbeat from core client for 30 sec - exiting
09:39:59 (1804): No heartbeat from core client for 30 sec - exiting
09:40:00 (1804): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:27:25 (1834): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:51:31 (1794): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:51:32 (1794): No heartbeat from core client for 30 sec - exiting
10:51:33 (1794): No heartbeat from core client for 30 sec - exiting
10:51:34 (1794): No heartbeat from core client for 30 sec - exiting
10:51:35 (1794): No heartbeat from core client for 30 sec - exiting
10:51:36 (1794): No heartbeat from core client for 30 sec - exiting
10:51:37 (1794): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0a165778 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6eb72)[0xf7493b72]
/lib32/libc.so.6(+0x6f812)[0xf7494812]
/lib32/libc.so.6(cfree+0x6d)[0xf74978cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf7694baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf7694c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf743e0f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:01 787619                             /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:01 787619                             /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
0a110000-0a175000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6fa7000-f7422000 rw-s 00000000 08:01 798454                             /var/lib/boinc-client/slots/3/138185
f7422000-f7425000 rw-p 00000000 00:00 0 
f7425000-f7598000 r-xp 00000000 08:01 792536                             /lib32/libc-2.13.so
f7598000-f7599000 ---p 00173000 08:01 792536                             /lib32/libc-2.13.so
f7599000-f759b000 r--p 00173000 08:01 792536                             /lib32/libc-2.13.so
f759b000-f759c000 rw-p 00175000 08:01 792536                             /lib32/libc-2.13.so
f759c000-f759f000 rw-p 00000000 00:00 0 
f759f000-f75bb000 r-xp 00000000 08:01 1179676                            /usr/lib32/libgcc_s.so.1
f75bb000-f75bc000 r--p 0001b000 08:01 1179676                            /usr/lib32/libgcc_s.so.1
f75bc000-f75bd000 rw-p 0001c000 08:01 1179676                            /usr/lib32/libgcc_s.so.1
f75bd000-f75e5000 r-xp 00000000 08:01 792553                             /lib32/libm-2.13.so
f75e5000-f75e6000 r--p 00028000 08:01 792553                             /lib32/libm-2.13.so
f75e6000-f75e7000 rw-p 00029000 08:01 792553                             /lib32/libm-2.13.so
f75e7000-f76c5000 r-xp 00000000 08:01 1182571                            /usr/lib32/libstdc++.so.6.0.16
f76c5000-f76c6000 ---p 000de000 08:01 1182571                            /usr/lib32/libstdc++.so.6.0.16
f76c6000-f76ca000 r--p 000de000 08:01 1182571                            /usr/lib32/libstdc++.so.6.0.16
f76ca000-f76cb000 rw-p 000e2000 08:01 1182571                            /usr/lib32/libstdc++.so.6.0.16
f76cb000-f76d3000 rw-p 00000000 00:00 0 
f76d3000-f76d6000 r-xp 00000000 08:01 792537                             /lib32/libdl-2.13.so
f76d6000-f76d7000 r--p 00002000 08:01 792537                             /lib32/libdl-2.13.so
f76d7000-f76d8000 rw-p 00003000 08:01 792537                             /lib32/libdl-2.13.so
f76d8000-f76ef000 r-xp 00000000 08:01 792551                             /lib32/libpthread-2.13.so
f76ef000-f76f0000 r--p 00016000 08:01 792551                             /lib32/libpthread-2.13.so
f76f0000-f76f1000 rw-p 00017000 08:01 792551                             /lib32/libpthread-2.13.so
f76f1000-f76f3000 rw-p 00000000 00:00 0 
f770f000-f7710000 rw-p 00000000 00:00 0 
f7710000-f7711000 ---p 00000000 00:00 0 
f7711000-f7714000 rw-p 00000000 00:00 0 
f7714000-f7716000 rw-s 00000000 08:01 798300                             /var/lib/boinc-client/slots/3/boinc_mmap_file
f7716000-f7718000 rw-p 00000000 00:00 0 
f7718000-f7719000 r-xp 00000000 00:00 0                                  [vdso]
f7719000-f7737000 r-xp 00000000 08:01 533260                             /lib/i386-linux-gnu/ld-2.13.so
f7737000-f7738000 r--p 0001d000 08:01 533260                             /lib/i386-linux-gnu/ld-2.13.so
f7738000-f7739000 rw-p 0001e000 08:01 533260                             /lib/i386-linux-gnu/ld-2.13.so
ffeb0000-fff20000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf7718400]
[0xf7718430]
/lib32/libc.so.6(gsignal+0x4f)[0xf7452c4f]
/lib32/libc.so.6(abort+0x175)[0xf7456175]
/lib32/libc.so.6(+0x63e8c)[0xf7488e8c]
/lib32/libc.so.6(+0x6eb72)[0xf7493b72]
/lib32/libc.so.6(+0x6f812)[0xf7494812]
/lib32/libc.so.6(cfree+0x6d)[0xf74978cd]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf7694baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf7694c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf743e0f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
19 Nov 2011 15:25:30 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 777,600 1,747,226 2.2469
18 Nov 2011 22:40:39 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 751,680 1,688,203 2.2459
18 Nov 2011 08:59:16 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 725,760 1,629,453 2.2452
18 Nov 2011 08:59:16 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 699,840 1,569,889 2.2432
18 Nov 2011 08:59:16 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 673,920 1,510,950 2.2420
18 Nov 2011 08:59:16 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 648,000 1,449,704 2.2372
18 Nov 2011 08:59:16 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 622,080 1,390,214 2.2348
18 Nov 2011 08:59:16 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 596,160 1,329,487 2.2301
08 Nov 2011 09:48:27 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 570,240 1,269,098 2.2256
05 Nov 2011 14:47:23 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 544,320 1,209,455 2.2220
04 Nov 2011 20:38:18 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 518,400 1,148,546 2.2156
03 Nov 2011 14:38:22 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 492,480 1,087,209 2.2076
01 Nov 2011 10:40:35 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 466,560 1,026,296 2.1997
31 Oct 2011 18:16:18 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 440,640 969,292 2.1997
31 Oct 2011 17:11:09 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 414,720 911,197 2.1971
31 Oct 2011 13:33:34 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 388,800 853,590 2.1954
31 Oct 2011 13:33:34 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 362,880 797,564 2.1979
31 Oct 2011 13:33:33 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 336,960 741,350 2.2001
31 Oct 2011 13:33:34 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 311,040 685,869 2.2051
18 Oct 2011 22:18:05 1132398 13334825 hadcm3n_o1cy_1900_40_007438994_0 285,120 629,042 2.2062


©2024 cpdn.org