Task 13023405

Name	hadcm3n_o5ym_1900_40_007203057_2
Workunit	7401337
Created	28 Jun 2011, 20:25:55 UTC
Sent	29 Jun 2011, 20:17:00 UTC
Report deadline	29 Sep 2011, 3:44:11 UTC
Received	25 Jul 2011, 20:07:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1127227
Run time	10 days 12 hours 56 min 57 sec
CPU time	10 days 12 hours 56 min 57 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.96 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/o5ymko.pja6c10 is not a valid UM file. Error converting file to netcdf: dataout/o5ymko.pja6c10 Error: Input file: dataout/o5ymko.pia6c10 is not a valid UM file. Error converting file to netcdf: dataout/o5ymko.pia6c10 Error: Input file: dataout/o5ymko.pfa6c10 is not a valid UM file. Error converting file to netcdf: dataout/o5ymko.pfa6c10 Error: Input file: dataout/o5ymka.pha6c10 is not a valid UM file. Error converting file to netcdf: dataout/o5ymka.pha6c10 Error: Input file: dataout/o5ymka.pga6c10 is not a valid UM file. Error converting file to netcdf: dataout/o5ymka.pga6c10 Error: Input file: dataout/o5ymka.pea6c10 is not a valid UM file. Error converting file to netcdf: dataout/o5ymka.pea6c10 Error: Input file: dataout/o5ymka.pda6c10 is not a valid UM file. Error converting file to netcdf: dataout/o5ymka.pda6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (!prev): 0x081fec18 *** ======= Backtrace: ========= /lib/libc.so.6(+0x6de2b)[0xf74a0e2b] /lib/libc.so.6(+0x6ebab)[0xf74a1bab] /lib/libc.so.6(cfree+0x6d)[0xf74a5a6d] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xf76976e1] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xf769773d] hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7449c2e] hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:03 4063644 /home/dc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:03 4063644 /home/dc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0823b000 rw-p 00000000 00:00 0 [heap] f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6fb6000-f7431000 rw-s 00000000 08:03 4064332 /home/dc/slots/3/136200 f7431000-f7433000 rw-p 00000000 00:00 0 f7433000-f7599000 r-xp 00000000 08:02 262953 /lib/libc-2.11.3.so f7599000-f759a000 ---p 00166000 08:02 262953 /lib/libc-2.11.3.so f759a000-f759c000 r--p 00166000 08:02 262953 /lib/libc-2.11.3.so f759c000-f759d000 rw-p 00168000 08:02 262953 /lib/libc-2.11.3.so f759d000-f75a0000 rw-p 00000000 00:00 0 f75a0000-f75bc000 r-xp 00000000 08:02 263006 /lib/libgcc_s.so.1 f75bc000-f75bd000 r--p 0001b000 08:02 263006 /lib/libgcc_s.so.1 f75bd000-f75be000 rw-p 0001c000 08:02 263006 /lib/libgcc_s.so.1 f75be000-f75bf000 rw-p 00000000 00:00 0 f75bf000-f75e7000 r-xp 00000000 08:02 262961 /lib/libm-2.11.3.so f75e7000-f75e8000 r--p 00027000 08:02 262961 /lib/libm-2.11.3.so f75e8000-f75e9000 rw-p 00028000 08:02 262961 /lib/libm-2.11.3.so f75e9000-f76cc000 r-xp 00000000 08:02 406565 /usr/lib/libstdc++.so.6.0.14 f76cc000-f76d0000 r--p 000e2000 08:02 406565 /usr/lib/libstdc++.so.6.0.14 f76d0000-f76d1000 rw-p 000e6000 08:02 406565 /usr/lib/libstdc++.so.6.0.14 f76d1000-f76d8000 rw-p 00000000 00:00 0 f76d8000-f76db000 r-xp 00000000 08:02 262959 /lib/libdl-2.11.3.so f76db000-f76dc000 r--p 00002000 08:02 262959 /lib/libdl-2.11.3.so f76dc000-f76dd000 rw-p 00003000 08:02 262959 /lib/libdl-2.11.3.so f76dd000-f76f4000 r-xp 00000000 08:02 262979 /lib/libpthread-2.11.3.so f76f4000-f76f5000 r--p 00016000 08:02 262979 /lib/libpthread-2.11.3.so f76f5000-f76f6000 rw-p 00017000 08:02 262979 /lib/libpthread-2.11.3.so f76f6000-f76f8000 rw-p 00000000 00:00 0 f770e000-f770f000 rw-p 00000000 00:00 0 f770f000-f7710000 ---p 00000000 00:00 0 f7710000-f7713000 rw-p 00000000 00:00 0 f7713000-f7715000 rw-s 00000000 08:03 4064232 /home/dc/slots/3/boinc_mmap_file f7715000-f7716000 rw-p 00000000 00:00 0 f7716000-f7735000 r-xp 00000000 08:02 262946 /lib/ld-2.11.3.so f7735000-f7736000 r--p 0001e000 08:02 262946 /lib/ld-2.11.3.so f7736000-f7737000 rw-p 0001f000 08:02 262946 /lib/ld-2.11.3.so ffd91000-ffe02000 rw-p 00000000 00:00 0 [stack] ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] SIGABRT: abort called Stack trace (19 frames): hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x4f)[0xf745e8df] /lib/libc.so.6(abort+0x180)[0xf7460220] /lib/libc.so.6(+0x67e07)[0xf749ae07] /lib/libc.so.6(+0x6de2b)[0xf74a0e2b] /lib/libc.so.6(+0x6ebab)[0xf74a1bab] /lib/libc.so.6(cfree+0x6d)[0xf74a5a6d] /usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xf76976e1] /usr/lib/libstdc++.so.6(_ZdaPv+0x1d)[0xf769773d] hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7449c2e] hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 23:16:11	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	259,200	910,615	3.5132
25 Jul 2011 20:40:47	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	233,280	822,327	3.5251
25 Jul 2011 19:01:24	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	207,360	733,401	3.5368
25 Jul 2011 17:28:53	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	181,440	645,483	3.5576
25 Jul 2011 14:25:48	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	155,520	556,751	3.5799
25 Jul 2011 14:25:48	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	129,600	464,440	3.5836
09 Jul 2011 20:47:00	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	103,680	374,104	3.6083
05 Jul 2011 23:17:21	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	77,760	277,639	3.5705
02 Jul 2011 10:11:51	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	51,840	184,985	3.5684
01 Jul 2011 07:13:06	1127227	13023405	hadcm3n_o5ym_1900_40_007203057_2	25,920	92,121	3.5541