Task 12910314

Name	hadcm3n_p6v6_1900_40_007226482_2
Workunit	7424722
Created	26 May 2011, 17:40:29 UTC
Sent	26 May 2011, 17:40:34 UTC
Report deadline	26 Aug 2011, 1:07:45 UTC
Received	8 Aug 2011, 16:09:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1143283
Run time	8 days 10 hours 10 min 18 sec
CPU time	7 days 17 hours 53 min 59 sec
Validate state	Invalid
Credit	4,976.64
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:33:35 (2237): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:33:37 (2237): No heartbeat from core client for 30 sec - exiting 00:33:38 (2237): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:17:06 (2233): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:14:28 (2307): No heartbeat from core client for 30 sec - exiting 19:14:30 (2307): No heartbeat from core client for 30 sec - exiting 19:14:31 (2307): No heartbeat from core client for 30 sec - exiting 19:14:32 (2307): No heartbeat from core client for 30 sec - exiting 19:14:33 (2307): No heartbeat from core client for 30 sec - exiting 19:14:34 (2307): No heartbeat from core client for 30 sec - exiting 19:14:35 (2307): No heartbeat from core client for 30 sec - exiting 19:14:36 (2307): No heartbeat from core client for 30 sec - exiting 19:14:37 (2307): No heartbeat from core client for 30 sec - exiting 19:14:38 (2307): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:02:25 (2470): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. 10:23:17 (2210): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. Signal 3 received, exiting... Called boinc_finish ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. 14:24:00 (2315): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. 17:14:57 (4041): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. 17:15:42 (5520): No heartbeat from core client for 30 sec - exiting 17:15:43 (5520): No heartbeat from core client for 30 sec - exiting 17:15:44 (5520): No heartbeat from core client for 30 sec - exiting 17:15:45 (5520): No heartbeat from core client for 30 sec - exiting 17:15:46 (5520): No heartbeat from core client for 30 sec - exiting 17:15:47 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2590, selfPID=2590, iMonCtr=1 ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. 18:12:38 (2262): No heartbeat from core client for 30 sec - exiting 18:12:39 (2262): No heartbeat from core client for 30 sec - exiting 18:12:40 (2262): No heartbeat from core client for 30 sec - exiting 18:12:41 (2262): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. 08:36:03 (2241): No heartbeat from core client for 30 sec - exiting 08:36:04 (2241): No heartbeat from core client for 30 sec - exiting 08:36:05 (2241): No heartbeat from core client for 30 sec - exiting 08:36:06 (2241): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. CPDN Monitor - Quit request from BOINC... ERROR: ld.so: object '/usr/lib/libc/memcpy-preload.so' from /etc/ld.so.preload cannot be preloaded: ignored. Signal 3 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf76ef400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804cedf] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050a03] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7447e46] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Jul 2011 15:28:52	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	414,720	648,408	1.5635
25 Jul 2011 22:15:07	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	388,800	614,029	1.5793
25 Jul 2011 21:05:18	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	362,880	573,134	1.5794
25 Jul 2011 19:38:35	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	336,960	532,192	1.5794
08 Jul 2011 18:58:33	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	311,040	491,151	1.5791
07 Jul 2011 15:36:24	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	285,120	450,322	1.5794
29 Jun 2011 17:20:11	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	259,200	409,333	1.5792
26 Jun 2011 17:05:18	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	233,280	368,362	1.5791
24 Jun 2011 22:12:55	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	207,360	327,338	1.5786
21 Jun 2011 20:55:54	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	181,440	286,320	1.5780
20 Jun 2011 17:21:21	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	155,520	245,342	1.5776
20 Jun 2011 17:21:21	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	129,600	204,393	1.5771
13 Jun 2011 20:21:46	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	103,680	163,481	1.5768
12 Jun 2011 21:04:54	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	77,760	122,556	1.5761
11 Jun 2011 14:42:27	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	51,840	81,607	1.5742
08 Jun 2011 20:18:46	1143283	12910314	hadcm3n_p6v6_1900_40_007226482_2	25,920	40,801	1.5741