climateprediction.net home page
Task 13809724

Task 13809724

Name hadcm3n_u05n_1980_40_007458039_4
Workunit 7655542
Created 22 Dec 2011, 21:32:42 UTC
Sent 22 Dec 2011, 21:58:20 UTC
Report deadline 23 Mar 2012, 5:25:31 UTC
Received 6 Jan 2012, 8:35:22 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 845875
Run time 11 days 1 hours 50 min 4 sec
CPU time 10 days 18 hours 28 min 9 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 1.02 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/u05nko.pji9c10 is not a valid UM file.
Error converting file to netcdf: dataout/u05nko.pji9c10
Error: Input file: dataout/u05nko.pii9c10 is not a valid UM file.
Error converting file to netcdf: dataout/u05nko.pii9c10
Error: Input file: dataout/u05nko.pfi9c10 is not a valid UM file.
Error converting file to netcdf: dataout/u05nko.pfi9c10
Error: Input file: dataout/u05nka.phi9c10 is not a valid UM file.
Error converting file to netcdf: dataout/u05nka.phi9c10
Error: Input file: dataout/u05nka.pgi9c10 is not a valid UM file.
Error converting file to netcdf: dataout/u05nka.pgi9c10
Error: Input file: dataout/u05nka.pei9c10 is not a valid UM file.
Error converting file to netcdf: dataout/u05nka.pei9c10
Error: Input file: dataout/u05nka.pdi9c10 is not a valid UM file.
Error converting file to netcdf: dataout/u05nka.pdi9c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0983fe98 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0x19a591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0x19bde8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0x19eecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0x5b4741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0x5b479d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x145bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
00110000-0012d000 r-xp 00000000 fc:00 4391007    /lib/libgcc_s.so.1
0012d000-0012e000 r--p 0001c000 fc:00 4391007    /lib/libgcc_s.so.1
0012e000-0012f000 rw-p 0001d000 fc:00 4391007    /lib/libgcc_s.so.1
0012f000-00282000 r-xp 00000000 fc:00 2818481    /lib/tls/i686/cmov/libc-2.11.1.so
00282000-00283000 ---p 00153000 fc:00 2818481    /lib/tls/i686/cmov/libc-2.11.1.so
00283000-00285000 r--p 00153000 fc:00 2818481    /lib/tls/i686/cmov/libc-2.11.1.so
00285000-00286000 rw-p 00155000 fc:00 2818481    /lib/tls/i686/cmov/libc-2.11.1.so
00286000-00289000 rw-p 00000000 00:00 0 
004f9000-005e2000 r-xp 00000000 fc:00 8372246    /usr/lib/libstdc++.so.6.0.13
005e2000-005e3000 ---p 000e9000 fc:00 8372246    /usr/lib/libstdc++.so.6.0.13
005e3000-005e7000 r--p 000e9000 fc:00 8372246    /usr/lib/libstdc++.so.6.0.13
005e7000-005e8000 rw-p 000ed000 fc:00 8372246    /usr/lib/libstdc++.so.6.0.13
005e8000-005ef000 rw-p 00000000 00:00 0 
00669000-0066b000 r-xp 00000000 fc:00 2818495    /lib/tls/i686/cmov/libdl-2.11.1.so
0066b000-0066c000 r--p 00001000 fc:00 2818495    /lib/tls/i686/cmov/libdl-2.11.1.so
0066c000-0066d000 rw-p 00002000 fc:00 2818495    /lib/tls/i686/cmov/libdl-2.11.1.so
00690000-006b4000 r-xp 00000000 fc:00 2818499    /lib/tls/i686/cmov/libm-2.11.1.so
006b4000-006b5000 r--p 00023000 fc:00 2818499    /lib/tls/i686/cmov/libm-2.11.1.so
006b5000-006b6000 rw-p 00024000 fc:00 2818499    /lib/tls/i686/cmov/libm-2.11.1.so
00aa2000-00aa3000 r-xp 00000000 00:00 0          [vdso]
00b8f000-00ba4000 r-xp 00000000 fc:00 2818523    /lib/tls/i686/cmov/libpthread-2.11.1.so
00ba4000-00ba5000 r--p 00014000 fc:00 2818523    /lib/tls/i686/cmov/libpthread-2.11.1.so
00ba5000-00ba6000 rw-p 00015000 fc:00 2818523    /lib/tls/i686/cmov/libpthread-2.11.1.so
00ba6000-00ba8000 rw-p 00000000 00:00 0 
00f3f000-00f5a000 r-xp 00000000 fc:00 2818084    /lib/ld-2.11.1.so
00f5a000-00f5b000 r--p 0001a000 fc:00 2818084    /lib/ld-2.11.1.so
00f5b000-00f5c000 rw-p 0001b000 fc:00 2818084    /lib/ld-2.11.1.so
08048000-080e3000 r-xp 00000000 fc:00 1867900    /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 fc:00 1867900    /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
097ea000-09850000 rw-p 00000000 00:00 0          [heap]
b7100000-b7121000 rw-p 00000000 00:00 0 
b7121000-b7200000 ---p 00000000 00:00 0 
b72fd000-b7778000 rw-s 00000000 fc:00 3522590    /var/lib/boinc-client/slots/0/134995
b7778000-b777b000 rw-p 00000000 00:00 0 
b778e000-b778f000 rw-p 00000000 00:00 0 
b778f000-b7790000 ---p 00000000 00:00 0 
b7790000-b7793000 rw-p 00000000 00:00 0 
b7793000-b7795000 rw-s 00000000 fc:00 3522587    /var/lib/boinc-client/slots/0/boinc_mmap_file
b7795000-b7797000 rw-p 00000000 00:00 0 
bfe34000-bfea5000 rw-p 00000000 00:00 0          [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xaa2400]
[0xaa2422]
/lib/tls/i686/cmov/libc.so.6(gsignal+0x51)[0x159651]
/lib/tls/i686/cmov/libc.so.6(abort+0x182)[0x15ca82]
/lib/tls/i686/cmov/libc.so.6(+0x6149d)[0x19049d]
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0x19a591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0x19bde8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0x19eecd]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0x5b4741]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0x5b479d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x145bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
06 Jan 2012 07:37:47 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 259,200 930,482 3.5898
02 Jan 2012 04:13:51 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 233,280 837,165 3.5887
31 Dec 2011 23:45:22 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 207,360 743,785 3.5869
30 Dec 2011 20:57:37 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 181,440 650,988 3.5879
29 Dec 2011 17:48:20 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 155,520 557,796 3.5867
28 Dec 2011 14:21:08 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 129,600 465,868 3.5947
27 Dec 2011 10:38:00 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 103,680 372,495 3.5927
26 Dec 2011 08:04:58 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 77,760 279,552 3.5951
25 Dec 2011 05:03:02 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 51,840 186,288 3.5935
24 Dec 2011 01:53:39 845875 13809724 hadcm3n_u05n_1980_40_007458039_4 25,920 92,859 3.5825


©2024 climateprediction.net