Task 15677385

Name	hadcm3n_3iky_1980_40_008334287_0
Workunit	8485148
Created	22 Mar 2013, 22:47:22 UTC
Sent	22 Mar 2013, 22:47:29 UTC
Report deadline	22 Jun 2013, 6:14:40 UTC
Received	5 May 2013, 1:41:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1260999
Run time	7 days 12 hours 21 min 31 sec
CPU time	6 days 18 hours 30 min 6 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.45 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.29</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08d91248 *** ======= Backtrace: ========= /lib/libc.so.6[0x4ea598c9] /lib/libstdc++.so.6(_ZdlPv+0x20)[0xf768eba0] /lib/libstdc++.so.6(_ZdaPv+0x1c)[0xf768ebfc] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf5)[0x4e9fd635] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:15 5250670 /var/lib/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:15 5250670 /var/lib/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08d3b000-08dc2000 rw-p 00000000 00:00 0 [heap] 4e9c1000-4e9e0000 r-xp 00000000 08:15 2101413 /usr/lib/ld-2.15.so 4e9e0000-4e9e1000 r--p 0001e000 08:15 2101413 /usr/lib/ld-2.15.so 4e9e1000-4e9e2000 rw-p 0001f000 08:15 2101413 /usr/lib/ld-2.15.so 4e9e4000-4eb8f000 r-xp 00000000 08:15 2132712 /usr/lib/libc-2.15.so 4eb8f000-4eb90000 ---p 001ab000 08:15 2132712 /usr/lib/libc-2.15.so 4eb90000-4eb92000 r--p 001ab000 08:15 2132712 /usr/lib/libc-2.15.so 4eb92000-4eb93000 rw-p 001ad000 08:15 2132712 /usr/lib/libc-2.15.so 4eb93000-4eb96000 rw-p 00000000 00:00 0 4eb98000-4eb9b000 r-xp 00000000 08:15 2108917 /usr/lib/libdl-2.15.so 4eb9b000-4eb9c000 r--p 00002000 08:15 2108917 /usr/lib/libdl-2.15.so 4eb9c000-4eb9d000 rw-p 00003000 08:15 2108917 /usr/lib/libdl-2.15.so 4eb9f000-4ebb5000 r-xp 00000000 08:15 2109104 /usr/lib/libpthread-2.15.so 4ebb5000-4ebb6000 r--p 00015000 08:15 2109104 /usr/lib/libpthread-2.15.so 4ebb6000-4ebb7000 rw-p 00016000 08:15 2109104 /usr/lib/libpthread-2.15.so 4ebb7000-4ebb9000 rw-p 00000000 00:00 0 4f00c000-4f035000 r-xp 00000000 08:15 2135513 /usr/lib/libm-2.15.so 4f035000-4f036000 r--p 00029000 08:15 2135513 /usr/lib/libm-2.15.so 4f036000-4f037000 rw-p 0002a000 08:15 2135513 /usr/lib/libm-2.15.so f71a9000-f71aa000 rw-p 00000000 00:00 0 f71aa000-f7625000 rw-s 00000000 08:15 7471269 /var/lib/boinc/slots/0/138130 f7625000-f7628000 rw-p 00000000 00:00 0 f7628000-f7644000 r-xp 00000000 08:15 2133527 /usr/lib/libgcc_s-4.7.2-20120921.so.1 f7644000-f7645000 rw-p 0001b000 08:15 2133527 /usr/lib/libgcc_s-4.7.2-20120921.so.1 f7645000-f7646000 rw-p 00000000 00:00 0 f7646000-f7723000 r-xp 00000000 08:15 2134029 /usr/lib/libstdc++.so.6.0.17 f7723000-f7727000 r--p 000dd000 08:15 2134029 /usr/lib/libstdc++.so.6.0.17 f7727000-f7728000 rw-p 000e1000 08:15 2134029 /usr/lib/libstdc++.so.6.0.17 f7728000-f772f000 rw-p 00000000 00:00 0 f775a000-f775b000 rw-p 00000000 00:00 0 f775b000-f775c000 ---p 00000000 00:00 0 f775c000-f775f000 rw-p 00000000 00:00 0 [stack:2168] f775f000-f7761000 rw-s 00000000 08:15 7471178 /var/lib/boinc/slots/0/boinc_mmap_file f7761000-f7762000 rw-p 00000000 00:00 0 f7762000-f7763000 r-xp 00000000 00:00 0 [vdso] ff853000-ff8c3000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf7762400] [0xf7762425] /lib/libc.so.6(gsignal+0x4f)[0x4ea1294f] /lib/libc.so.6(abort+0x143)[0x4ea142a3] /lib/libc.so.6[0x4ea51ed5] /lib/libc.so.6[0x4ea598c9] /lib/libstdc++.so.6(_ZdlPv+0x20)[0xf768eba0] /lib/libstdc++.so.6(_ZdaPv+0x1c)[0xf768ebfc] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf5)[0x4e9fd635] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 May 2013 01:43:03	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	259,200	584,999	2.2569
28 Apr 2013 03:39:31	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	233,280	525,319	2.2519
21 Apr 2013 09:07:11	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	207,360	466,045	2.2475
20 Apr 2013 13:25:31	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	181,440	406,573	2.2408
14 Apr 2013 12:22:44	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	155,520	347,259	2.2329
08 Apr 2013 04:10:25	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	129,600	287,864	2.2212
07 Apr 2013 00:25:03	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	103,680	228,391	2.2028
01 Apr 2013 05:18:19	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	77,760	177,844	2.2871
30 Mar 2013 03:22:40	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	51,840	118,686	2.2895
23 Mar 2013 18:45:59	1260999	15677385	hadcm3n_3iky_1980_40_008334287_0	25,920	59,318	2.2885