Task 15724973

Name	hadcm3n_ze7i_1920_40_008281712_1
Workunit	8432847
Created	15 Apr 2013, 7:37:15 UTC
Sent	15 Apr 2013, 7:37:42 UTC
Report deadline	15 Jul 2013, 15:04:53 UTC
Received	30 May 2013, 8:37:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1268906
Run time	20 days 12 hours 15 min 8 sec
CPU time	19 days 4 hours 18 min 2 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.19 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:38:53 (3151): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x090cf200 *** ======= Backtrace: ========= /lib/libc.so.6[0x47fdcff9] /lib/libstdc++.so.6(_ZdlPv+0x20)[0x485d0500] /lib/libstdc++.so.6(_ZdaPv+0x1c)[0x485d055c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf5)[0x47f80865] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 fd:01 659994 /var/lib/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 fd:01 659994 /var/lib/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 09079000-09100000 rw-p 00000000 00:00 0 [heap] 47f44000-47f63000 r-xp 00000000 fd:01 1051405 /usr/lib/ld-2.16.so 47f63000-47f64000 r--p 0001e000 fd:01 1051405 /usr/lib/ld-2.16.so 47f64000-47f65000 rw-p 0001f000 fd:01 1051405 /usr/lib/ld-2.16.so 47f67000-48117000 r-xp 00000000 fd:01 1073727 /usr/lib/libc-2.16.so 48117000-48119000 r--p 001b0000 fd:01 1073727 /usr/lib/libc-2.16.so 48119000-4811a000 rw-p 001b2000 fd:01 1073727 /usr/lib/libc-2.16.so 4811a000-4811d000 rw-p 00000000 00:00 0 4811f000-48135000 r-xp 00000000 fd:01 1073728 /usr/lib/libpthread-2.16.so 48135000-48136000 r--p 00015000 fd:01 1073728 /usr/lib/libpthread-2.16.so 48136000-48137000 rw-p 00016000 fd:01 1073728 /usr/lib/libpthread-2.16.so 48137000-48139000 rw-p 00000000 00:00 0 4813b000-4813e000 r-xp 00000000 fd:01 1049405 /usr/lib/libdl-2.16.so 4813e000-4813f000 r--p 00002000 fd:01 1049405 /usr/lib/libdl-2.16.so 4813f000-48140000 rw-p 00003000 fd:01 1049405 /usr/lib/libdl-2.16.so 4814d000-4818b000 r-xp 00000000 fd:01 1069281 /usr/lib/libm-2.16.so 4818b000-4818c000 r--p 0003d000 fd:01 1069281 /usr/lib/libm-2.16.so 4818c000-4818d000 rw-p 0003e000 fd:01 1069281 /usr/lib/libm-2.16.so 48526000-48542000 r-xp 00000000 fd:01 1062016 /usr/lib/libgcc_s-4.7.2-20121109.so.1 48542000-48543000 r--p 0001b000 fd:01 1062016 /usr/lib/libgcc_s-4.7.2-20121109.so.1 48543000-48544000 rw-p 0001c000 fd:01 1062016 /usr/lib/libgcc_s-4.7.2-20121109.so.1 48585000-48665000 r-xp 00000000 fd:01 1072352 /usr/lib/libstdc++.so.6.0.17 48665000-48669000 r--p 000df000 fd:01 1072352 /usr/lib/libstdc++.so.6.0.17 48669000-4866b000 rw-p 000e3000 fd:01 1072352 /usr/lib/libstdc++.so.6.0.17 4866b000-48671000 rw-p 00000000 00:00 0 f731b000-f7796000 rw-s 00000000 fd:01 656976 /var/lib/boinc/slots/9/135945 f7796000-f779a000 rw-p 00000000 00:00 0 f77af000-f77b1000 rw-p 00000000 00:00 0 f77b1000-f77b2000 ---p 00000000 00:00 0 f77b2000-f77b5000 rw-p 00000000 00:00 0 [stack:2546] f77b5000-f77b7000 rw-s 00000000 fd:01 656626 /var/lib/boinc/slots/9/boinc_mmap_file f77b7000-f77b8000 rw-p 00000000 00:00 0 f77b8000-f77b9000 r-xp 00000000 00:00 0 [vdso] ffe20000-ffe90000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf77b8400] [0xf77b8430] /lib/libc.so.6(gsignal+0x4f)[0x47f95b7f] /lib/libc.so.6(abort+0x143)[0x47f974d3] /lib/libc.so.6[0x47fd5405] /lib/libc.so.6[0x47fdcff9] /lib/libstdc++.so.6(_ZdlPv+0x20)[0x485d0500] /lib/libstdc++.so.6(_ZdaPv+0x1c)[0x485d055c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf5)[0x47f80865] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 May 2013 08:37:54	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	1,036,800	1,657,074	1.5983
28 May 2013 10:29:17	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	1,010,880	1,615,552	1.5982
27 May 2013 16:52:12	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	984,960	1,573,658	1.5977
27 May 2013 03:57:14	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	959,040	1,532,595	1.5981
26 May 2013 10:50:46	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	933,120	1,490,013	1.5968
25 May 2013 17:40:12	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	907,200	1,447,834	1.5959
25 May 2013 04:10:38	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	881,280	1,408,150	1.5978
23 May 2013 08:53:09	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	855,360	1,362,122	1.5925
22 May 2013 10:33:58	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	829,440	1,315,190	1.5856
21 May 2013 04:26:23	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	803,520	1,273,764	1.5852
19 May 2013 21:10:50	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	777,600	1,232,276	1.5847
18 May 2013 23:08:31	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	751,680	1,190,762	1.5841
18 May 2013 07:18:04	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	725,760	1,147,634	1.5813
16 May 2013 22:25:54	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	699,840	1,106,935	1.5817
16 May 2013 03:45:27	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	673,920	1,066,012	1.5818
14 May 2013 09:59:12	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	648,000	1,025,330	1.5823
13 May 2013 10:09:14	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	622,080	984,851	1.5832
12 May 2013 18:45:14	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	596,160	944,055	1.5836
11 May 2013 11:19:01	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	570,240	901,753	1.5814
10 May 2013 06:00:23	1268906	15724973	hadcm3n_ze7i_1920_40_008281712_1	544,320	861,652	1.5830