climateprediction.net home page
Task 14280643

Task 14280643

Name hadcm3n_t2yk_1940_40_007831662_3
Workunit 7986774
Created 17 Mar 2012, 22:59:13 UTC
Sent 17 Mar 2012, 22:59:28 UTC
Report deadline 17 Jun 2012, 6:26:39 UTC
Received 29 Mar 2012, 10:56:45 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1185009
Run time 6 days 14 hours 54 min 39 sec
CPU time 6 days 11 hours 5 min 40 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 2.44 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.34</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
23:05:02 (27616): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:01:27 (31960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:03:40 (5175): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:42:57 (5302): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:44:40 (12986): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:54:03 (13037): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:55:11 (13212): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:10:26 (13273): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:12:03 (14274): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 15 received, exiting...
Called boinc_finish
00:29:35 (2033): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:47:38 (14394): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:50:35 (17174): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:04:54 (17424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:44:59 (17929): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:55:17 (18066): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:52:21 (18132): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:16:33 (8245): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
02:18:37 (8292): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
02:22:05 (8328): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:28:58 (8360): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:37:39 (8393): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:38:54 (8431): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:39:52 (8463): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:28:25 (8495): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:42:29 (8529): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:06:02 (8561): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:46:44 (8594): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:47:57 (8629): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:49:29 (8661): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:52:02 (8683): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:53:18 (8724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:59:46 (19276): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:02:10 (2624): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:03:59 (2654): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:07:19 (2688): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:55:07 (2722): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x091d8e18 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6bff1)[0xf7532ff1]
/lib32/libc.so.6(+0x6d880)[0xf7534880]
/lib32/libc.so.6(cfree+0x6d)[0xf753792d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf770e7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf770e80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74ddce7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:01 26740438                           /usr/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:01 26740438                           /usr/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09183000-091e8000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f7049000-f74c4000 rw-s 00000000 08:01 26740773                           /usr/BOINC/slots/4/137515
f74c4000-f74c7000 rw-p 00000000 00:00 0 
f74c7000-f761b000 r-xp 00000000 08:01 6291607                            /lib32/libc-2.12.1.so
f761b000-f761c000 ---p 00154000 08:01 6291607                            /lib32/libc-2.12.1.so
f761c000-f761e000 r--p 00154000 08:01 6291607                            /lib32/libc-2.12.1.so
f761e000-f761f000 rw-p 00156000 08:01 6291607                            /lib32/libc-2.12.1.so
f761f000-f7622000 rw-p 00000000 00:00 0 
f7622000-f763c000 r-xp 00000000 08:01 23074874                           /usr/lib32/libgcc_s.so.1
f763c000-f763d000 r--p 00019000 08:01 23074874                           /usr/lib32/libgcc_s.so.1
f763d000-f763e000 rw-p 0001a000 08:01 23074874                           /usr/lib32/libgcc_s.so.1
f763e000-f7662000 r-xp 00000000 08:01 6291611                            /lib32/libm-2.12.1.so
f7662000-f7663000 r--p 00023000 08:01 6291611                            /lib32/libm-2.12.1.so
f7663000-f7664000 rw-p 00024000 08:01 6291611                            /lib32/libm-2.12.1.so
f7664000-f7743000 r-xp 00000000 08:01 23074877                           /usr/lib32/libstdc++.so.6.0.14
f7743000-f7747000 r--p 000de000 08:01 23074877                           /usr/lib32/libstdc++.so.6.0.14
f7747000-f7748000 rw-p 000e2000 08:01 23074877                           /usr/lib32/libstdc++.so.6.0.14
f7748000-f774f000 rw-p 00000000 00:00 0 
f774f000-f7751000 r-xp 00000000 08:01 6291610                            /lib32/libdl-2.12.1.so
f7751000-f7752000 r--p 00001000 08:01 6291610                            /lib32/libdl-2.12.1.so
f7752000-f7753000 rw-p 00002000 08:01 6291610                            /lib32/libdl-2.12.1.so
f7753000-f7754000 rw-p 00000000 00:00 0 
f7754000-f7769000 r-xp 00000000 08:01 6291621                            /lib32/libpthread-2.12.1.so
f7769000-f776a000 r--p 00014000 08:01 6291621                            /lib32/libpthread-2.12.1.so
f776a000-f776b000 rw-p 00015000 08:01 6291621                            /lib32/libpthread-2.12.1.so
f776b000-f776d000 rw-p 00000000 00:00 0 
f7781000-f7782000 rw-p 00000000 00:00 0 
f7782000-f7783000 ---p 00000000 00:00 0 
f7783000-f7786000 rw-p 00000000 00:00 0 
f7786000-f7788000 rw-s 00000000 08:01 26740769                           /usr/BOINC/slots/4/boinc_mmap_file
f7788000-f778a000 rw-p 00000000 00:00 0 
f778a000-f778b000 r-xp 00000000 00:00 0                                  [vdso]
f778b000-f77a7000 r-xp 00000000 08:01 6291604                            /lib32/ld-2.12.1.so
f77a7000-f77a8000 r--p 0001b000 08:01 6291604                            /lib32/ld-2.12.1.so
f77a8000-f77a9000 rw-p 0001c000 08:01 6291604                            /lib32/ld-2.12.1.so
ffc0c000-ffc7d000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf778a400]
[0xf778a430]
/lib32/libc.so.6(gsignal+0x51)[0xf74f1a01]
/lib32/libc.so.6(abort+0x182)[0xf74f4e42]
/lib32/libc.so.6(+0x61f15)[0xf7528f15]
/lib32/libc.so.6(+0x6bff1)[0xf7532ff1]
/lib32/libc.so.6(+0x6d880)[0xf7534880]
/lib32/libc.so.6(cfree+0x6d)[0xf753792d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf770e7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf770e80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74ddce7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
29 Mar 2012 10:59:07 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 259,200 558,334 2.1541
28 Mar 2012 22:02:57 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 233,280 503,294 2.1575
28 Mar 2012 02:09:42 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 207,360 446,744 2.1544
27 Mar 2012 21:59:06 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 181,440 390,351 2.1514
26 Mar 2012 22:00:55 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 155,520 335,362 2.1564
26 Mar 2012 01:38:12 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 129,600 278,965 2.1525
25 Mar 2012 22:04:00 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 103,680 222,844 2.1493
24 Mar 2012 23:03:54 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 77,760 167,078 2.1486
24 Mar 2012 02:14:25 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 51,840 111,823 2.1571
23 Mar 2012 23:00:50 1185009 14280643 hadcm3n_t2yk_1940_40_007831662_3 25,920 56,164 2.1668


©2024 climateprediction.net