Task 16060723

Name	hadcm3n_o8u1_1900_40_008466572_1
Workunit	8617411
Created	7 Oct 2013, 22:30:38 UTC
Sent	7 Oct 2013, 22:56:18 UTC
Report deadline	7 Jan 2014, 6:23:29 UTC
Received	13 Dec 2013, 12:18:02 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID	1228872
Run time	34 days 2 hours 50 min 41 sec
CPU time	16 days 8 hours 19 min 9 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x089b95d8 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08d0adc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x088ecdc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09cf2dc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09d18dc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x085abdc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08bdedc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x086c1dc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x081e4dc0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:18:14 (2896): No heartbeat from core client for 30 sec - exiting 09:18:15 (2896): No heartbeat from core client for 30 sec - exiting 09:18:16 (2896): No heartbeat from core client for 30 sec - exiting 09:18:17 (2896): No heartbeat from core client for 30 sec - exiting 09:18:18 (2896): No heartbeat from core client for 30 sec - exiting 09:18:19 (2896): No heartbeat from core client for 30 sec - exiting 09:18:20 (2896): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Oct 2013 19:12:50	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	777,600	1,408,123	1.8109
26 Oct 2013 00:23:17	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	751,680	1,359,332	1.8084
25 Oct 2013 06:22:03	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	725,760	1,311,151	1.8066
24 Oct 2013 17:14:21	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	699,840	1,265,732	1.8086
24 Oct 2013 01:05:09	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	673,920	1,217,253	1.8062
23 Oct 2013 08:36:40	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	648,000	1,167,926	1.8024
22 Oct 2013 09:58:26	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	622,080	1,121,454	1.8027
21 Oct 2013 18:37:07	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	596,160	1,076,059	1.8050
21 Oct 2013 05:32:40	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	570,240	1,030,586	1.8073
20 Oct 2013 13:37:22	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	544,320	982,608	1.8052
19 Oct 2013 20:59:03	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	518,400	933,360	1.8005
19 Oct 2013 04:39:18	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	492,480	887,995	1.8031
18 Oct 2013 11:51:17	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	466,560	840,860	1.8023
17 Oct 2013 16:47:53	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	440,640	792,341	1.7982
17 Oct 2013 03:35:57	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	414,720	746,818	1.8008
16 Oct 2013 12:21:10	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	388,800	701,364	1.8039
15 Oct 2013 22:01:16	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	362,880	655,447	1.8062
15 Oct 2013 05:34:06	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	336,960	609,104	1.8076
14 Oct 2013 16:30:54	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	311,040	563,633	1.8121
14 Oct 2013 02:59:24	1228872	16060723	hadcm3n_o8u1_1900_40_008466572_1	285,120	516,392	1.8111