Task 15454043

Name	hadcm3n_zhgb_1880_40_008251308_2
Workunit	8406432
Created	22 Nov 2012, 16:03:00 UTC
Sent	22 Nov 2012, 16:03:10 UTC
Report deadline	21 Feb 2013, 23:30:21 UTC
Received	15 Feb 2013, 7:47:01 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID	1219037
Run time	11 days 7 hours 14 min 34 sec
CPU time	4 days 5 hours 53 min 28 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.96 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.36</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> . Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x085b50f0 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 11:06:22 (7035): No heartbeat from core client for 30 sec - exiting 11:06:23 (7035): No heartbeat from core client for 30 sec - exiting 11:06:24 (7035): No heartbeat from core client for 30 sec - exiting 11:06:25 (7035): No heartbeat from core client for 30 sec - exiting 11:06:26 (7035): No heartbeat from core client for 30 sec - exiting 11:06:27 (7035): No heartbeat from core client for 30 sec - exiting 11:15:09 (7035): No heartbeat from core client for 30 sec - exiting 11:15:10 (7035): No heartbeat from core client for 30 sec - exiting * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0941d888 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x097af888 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 16:34:49 (2940): No heartbeat from core client for 30 sec - exiting 16:34:50 (2940): No heartbeat from core client for 30 sec - exiting 16:34:51 (2940): No heartbeat from core client for 30 sec - exiting 16:34:52 (2940): No heartbeat from core client for 30 sec - exiting 16:34:53 (2940): No heartbeat from core client for 30 sec - exiting 16:34:54 (2940): No heartbeat from core client for 30 sec - exiting 16:34:55 (2940): No heartbeat from core client for 30 sec - exiting 16:34:56 (2940): No heartbeat from core client for 30 sec - exiting 16:34:57 (2940): No heartbeat from core client for 30 sec - exiting 16:34:58 (2940): No heartbeat from core client for 30 sec - exiting 16:34:59 (2940): No heartbeat from core client for 30 sec - exiting 00:43:46 (2940): No heartbeat from core client for 30 sec - exiting 00:43:47 (2940): No heartbeat from core client for 30 sec - exiting 00:43:48 (2940): No heartbeat from core client for 30 sec - exiting 00:43:49 (2940): No heartbeat from core client for 30 sec - exiting 00:43:50 (2940): No heartbeat from core client for 30 sec - exiting 00:43:51 (2940): No heartbeat from core client for 30 sec - exiting 15:13:27 (2940): No heartbeat from core client for 30 sec - exiting 15:13:28 (2940): No heartbeat from core client for 30 sec - exiting 15:13:29 (2940): No heartbeat from core client for 30 sec - exiting 15:13:30 (2940): No heartbeat from core client for 30 sec - exiting 15:13:31 (2940): No heartbeat from core client for 30 sec - exiting 15:13:32 (2940): No heartbeat from core client for 30 sec - exiting 15:13:33 (2940): No heartbeat from core client for 30 sec - exiting 15:13:34 (2940): No heartbeat from core client for 30 sec - exiting 15:13:35 (2940): No heartbeat from core client for 30 sec - exiting 15:13:36 (2940): No heartbeat from core client for 30 sec - exiting 15:13:37 (2940): No heartbeat from core client for 30 sec - exiting 09:20:27 (2940): No heartbeat from core client for 30 sec - exiting 09:20:28 (2940): No heartbeat from core client for 30 sec - exiting 09:20:29 (2940): No heartbeat from core client for 30 sec - exiting 09:20:30 (2940): No heartbeat from core client for 30 sec - exiting 09:20:31 (2940): No heartbeat from core client for 30 sec - exiting 09:20:32 (2940): No heartbeat from core client for 30 sec - exiting 00:33:11 (2940): No heartbeat from core client for 30 sec - exiting 00:33:12 (2940): No heartbeat from core client for 30 sec - exiting 00:33:13 (2940): No heartbeat from core client for 30 sec - exiting 00:33:14 (2940): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08fcf888 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 11:01:27 (3144): No heartbeat from core client for 30 sec - exiting 16:04:30 (3144): No heartbeat from core client for 30 sec - exiting * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09b36888 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 10:30:51 (2244): No heartbeat from core client for 30 sec - exiting 14:24:26 (2244): No heartbeat from core client for 30 sec - exiting 14:24:27 (2244): No heartbeat from core client for 30 sec - exiting 14:24:28 (2244): No heartbeat from core client for 30 sec - exiting 14:24:29 (2244): No heartbeat from core client for 30 sec - exiting 14:24:30 (2244): No heartbeat from core client for 30 sec - exiting 14:24:31 (2244): No heartbeat from core client for 30 sec - exiting 14:24:32 (2244): No heartbeat from core client for 30 sec - exiting 14:24:33 (2244): No heartbeat from core client for 30 sec - exiting 14:24:34 (2244): No heartbeat from core client for 30 sec - exiting 14:24:35 (2244): No heartbeat from core client for 30 sec - exiting 12:43:20 (2244): No heartbeat from core client for 30 sec - exiting 00:15:28 (2244): No heartbeat from core client for 30 sec - exiting 00:15:29 (2244): No heartbeat from core client for 30 sec - exiting 00:15:30 (2244): No heartbeat from core client for 30 sec - exiting 00:15:31 (2244): No heartbeat from core client for 30 sec - exiting 00:15:32 (2244): No heartbeat from core client for 30 sec - exiting 01:37:47 (2244): No heartbeat from core client for 30 sec - exiting 01:37:48 (2244): No heartbeat from core client for 30 sec - exiting 01:37:49 (2244): No heartbeat from core client for 30 sec - exiting 01:37:50 (2244): No heartbeat from core client for 30 sec - exiting 01:37:51 (2244): No heartbeat from core client for 30 sec - exiting 11:24:21 (2244): No heartbeat from core client for 30 sec - exiting 11:24:22 (2244): No heartbeat from core client for 30 sec - exiting 11:24:23 (2244): No heartbeat from core client for 30 sec - exiting 11:24:24 (2244): No heartbeat from core client for 30 sec - exiting 11:24:25 (2244): No heartbeat from core client for 30 sec - exiting 11:24:26 (2244): No heartbeat from core client for 30 sec - exiting 11:24:27 (2244): No heartbeat from core client for 30 sec - exiting 11:24:28 (2244): No heartbeat from core client for 30 sec - exiting 11:24:29 (2244): No heartbeat from core client for 30 sec - exiting 11:24:30 (2244): No heartbeat from core client for 30 sec - exiting 11:24:31 (2244): No heartbeat from core client for 30 sec - exiting 11:24:32 (2244): No heartbeat from core client for 30 sec - exiting 11:24:33 (2244): No heartbeat from core client for 30 sec - exiting 11:24:34 (2244): No heartbeat from core client for 30 sec - exiting 11:24:35 (2244): No heartbeat from core client for 30 sec - exiting 11:24:36 (2244): No heartbeat from core client for 30 sec - exiting 11:24:37 (2244): No heartbeat from core client for 30 sec - exiting 11:24:38 (2244): No heartbeat from core client for 30 sec - exiting 11:24:39 (2244): No heartbeat from core client for 30 sec - exiting 11:24:40 (2244): No heartbeat from core client for 30 sec - exiting 11:24:41 (2244): No heartbeat from core client for 30 sec - exiting 11:24:42 (2244): No heartbeat from core client for 30 sec - exiting 11:24:43 (2244): No heartbeat from core client for 30 sec - exiting 11:24:44 (2244): No heartbeat from core client for 30 sec - exiting 11:24:45 (2244): No heartbeat from core client for 30 sec - exiting 11:24:46 (2244): No heartbeat from core client for 30 sec - exiting 11:24:47 (2244): No heartbeat from core client for 30 sec - exiting 11:24:48 (2244): No heartbeat from core client for 30 sec - exiting 11:24:49 (2244): No heartbeat from core client for 30 sec - exiting 11:24:50 (2244): No heartbeat from core client for 30 sec - exiting 11:24:51 (2244): No heartbeat from core client for 30 sec - exiting 11:36:57 (2244): No heartbeat from core client for 30 sec - exiting 00:17:45 (2244): No heartbeat from core client for 30 sec - exiting 00:17:46 (2244): No heartbeat from core client for 30 sec - exiting 00:17:47 (2244): No heartbeat from core client for 30 sec - exiting 00:17:48 (2244): No heartbeat from core client for 30 sec - exiting 00:17:49 (2244): No heartbeat from core client for 30 sec - exiting 00:17:50 (2244): No heartbeat from core client for 30 sec - exiting 00:17:51 (2244): No heartbeat from core client for 30 sec - exiting 00:17:52 (2244): No heartbeat from core client for 30 sec - exiting 00:17:53 (2244): No heartbeat from core client for 30 sec - exiting 00:17:54 (2244): No heartbeat from core client for 30 sec - exiting 23:42:44 (2244): No heartbeat from core client for 30 sec - exiting 23:42:45 (2244): No heartbeat from core client for 30 sec - exiting 23:42:46 (2244): No heartbeat from core client for 30 sec - exiting 23:42:47 (2244): No heartbeat from core client for 30 sec - exiting 23:42:48 (2244): No heartbeat from core client for 30 sec - exiting 23:42:49 (2244): No heartbeat from core client for 30 sec - exiting 23:42:50 (2244): No heartbeat from core client for 30 sec - exiting 23:42:51 (2244): No heartbeat from core client for 30 sec - exiting 23:42:52 (2244): No heartbeat from core client for 30 sec - exiting 23:42:53 (2244): No heartbeat from core client for 30 sec - exiting 02:49:18 (2244): No heartbeat from core client for 30 sec - exiting 01:42:13 (2244): No heartbeat from core client for 30 sec - exiting 00:23:56 (2244): No heartbeat from core client for 30 sec - exiting 00:23:57 (2244): No heartbeat from core client for 30 sec - exiting 00:23:58 (2244): No heartbeat from core client for 30 sec - exiting 00:23:59 (2244): No heartbeat from core client for 30 sec - exiting 00:24:00 (2244): No heartbeat from core client for 30 sec - exiting 00:24:01 (2244): No heartbeat from core client for 30 sec - exiting 00:24:02 (2244): No heartbeat from core client for 30 sec - exiting 00:24:03 (2244): No heartbeat from core client for 30 sec - exiting 00:24:04 (2244): No heartbeat from core client for 30 sec - exiting 00:24:05 (2244): No heartbeat from core client for 30 sec - exiting 00:31:45 (2244): No heartbeat from core client for 30 sec - exiting 04:54:49 (2244): No heartbeat from core client for 30 sec - exiting 04:54:50 (2244): No heartbeat from core client for 30 sec - exiting 04:54:51 (2244): No heartbeat from core client for 30 sec - exiting 04:54:52 (2244): No heartbeat from core client for 30 sec - exiting 04:54:53 (2244): No heartbeat from core client for 30 sec - exiting 04:54:54 (2244): No heartbeat from core client for 30 sec - exiting 05:55:30 (2244): No heartbeat from core client for 30 sec - exiting 05:55:31 (2244): No heartbeat from core client for 30 sec - exiting 05:55:32 (2244): No heartbeat from core client for 30 sec - exiting 05:55:33 (2244): No heartbeat from core client for 30 sec - exiting 05:55:34 (2244): No heartbeat from core client for 30 sec - exiting 05:55:35 (2244): No heartbeat from core client for 30 sec - exiting 09:11:06 (2244): No heartbeat from core client for 30 sec - exiting 09:11:07 (2244): No heartbeat from core client for 30 sec - exiting 09:11:08 (2244): No heartbeat from core client for 30 sec - exiting 09:11:09 (2244): No heartbeat from core client for 30 sec - exiting 09:11:10 (2244): No heartbeat from core client for 30 sec - exiting 09:11:11 (2244): No heartbeat from core client for 30 sec - exiting 09:11:12 (2244): No heartbeat from core client for 30 sec - exiting 09:11:13 (2244): No heartbeat from core client for 30 sec - exiting 09:11:14 (2244): No heartbeat from core client for 30 sec - exiting 09:11:15 (2244): No heartbeat from core client for 30 sec - exiting 09:11:16 (2244): No heartbeat from core client for 30 sec - exiting 09:23:47 (2244): No heartbeat from core client for 30 sec - exiting 09:23:48 (2244): No heartbeat from core client for 30 sec - exiting 09:23:49 (2244): No heartbeat from core client for 30 sec - exiting 09:23:50 (2244): No heartbeat from core client for 30 sec - exiting 09:23:51 (2244): No heartbeat from core client for 30 sec - exiting 09:23:52 (2244): No heartbeat from core client for 30 sec - exiting 09:23:53 (2244): No heartbeat from core client for 30 sec - exiting 09:23:54 (2244): No heartbeat from core client for 30 sec - exiting 09:23:55 (2244): No heartbeat from core client for 30 sec - exiting 09:23:56 (2244): No heartbeat from core client for 30 sec - exiting 09:23:57 (2244): No heartbeat from core client for 30 sec - exiting 23:58:20 (2244): No heartbeat from core client for 30 sec - exiting 23:58:21 (2244): No heartbeat from core client for 30 sec - exiting 23:58:22 (2244): No heartbeat from core client for 30 sec - exiting 23:58:23 (2244): No heartbeat from core client for 30 sec - exiting 23:58:25 (2244): No heartbeat from core client for 30 sec - exiting 23:58:26 (2244): No heartbeat from core client for 30 sec - exiting 23:58:27 (2244): No heartbeat from core client for 30 sec - exiting 23:58:28 (2244): No heartbeat from core client for 30 sec - exiting 23:58:29 (2244): No heartbeat from core client for 30 sec - exiting 23:58:30 (2244): No heartbeat from core client for 30 sec - exiting 03:02:53 (2244): No heartbeat from core client for 30 sec - exiting 03:02:54 (2244): No heartbeat from core client for 30 sec - exiting 03:02:55 (2244): No heartbeat from core client for 30 sec - exiting 03:02:56 (2244): No heartbeat from core client for 30 sec - exiting 03:02:57 (2244): No heartbeat from core client for 30 sec - exiting 03:02:58 (2244): No heartbeat from core client for 30 sec - exiting 03:02:59 (2244): No heartbeat from core client for 30 sec - exiting 03:03:00 (2244): No heartbeat from core client for 30 sec - exiting 03:03:01 (2244): No heartbeat from core client for 30 sec - exiting 03:03:02 (2244): No heartbeat from core client for 30 sec - exiting 03:05:54 (2244): No heartbeat from core client for 30 sec - exiting 03:09:04 (2244): No heartbeat from core client for 30 sec - exiting 03:09:05 (2244): No heartbeat from core client for 30 sec - exiting 03:09:06 (2244): No heartbeat from core client for 30 sec - exiting 03:09:07 (2244): No heartbeat from core client for 30 sec - exiting 03:09:08 (2244): No heartbeat from core client for 30 sec - exiting 03:09:09 (2244): No heartbeat from core client for 30 sec - exiting 03:19:38 (2244): No heartbeat from core client for 30 sec - exiting 03:19:39 (2244): No heartbeat from core client for 30 sec - exiting 03:19:40 (2244): No heartbeat from core client for 30 sec - exiting 03:19:41 (2244): No heartbeat from core client for 30 sec - exiting 03:19:42 (2244): No heartbeat from core client for 30 sec - exiting 03:19:43 (2244): No heartbeat from core client for 30 sec - exiting 03:19:44 (2244): No heartbeat from core client for 30 sec - exiting 03:19:45 (2244): No heartbeat from core client for 30 sec - exiting 03:19:46 (2244): No heartbeat from core client for 30 sec - exiting 03:19:47 (2244): No heartbeat from core client for 30 sec - exiting 03:19:48 (2244): No heartbeat from core client for 30 sec - exiting 03:50:56 (2244): No heartbeat from core client for 30 sec - exiting 03:50:57 (2244): No heartbeat from core client for 30 sec - exiting 03:50:58 (2244): No heartbeat from core client for 30 sec - exiting 03:50:59 (2244): No heartbeat from core client for 30 sec - exiting 03:51:00 (2244): No heartbeat from core client for 30 sec - exiting 03:51:01 (2244): No heartbeat from core client for 30 sec - exiting 03:57:21 (2244): No heartbeat from core client for 30 sec - exiting 03:57:22 (2244): No heartbeat from core client for 30 sec - exiting 03:57:23 (2244): No heartbeat from core client for 30 sec - exiting 03:57:24 (2244): No heartbeat from core client for 30 sec - exiting 03:57:25 (2244): No heartbeat from core client for 30 sec - exiting 03:57:26 (2244): No heartbeat from core client for 30 sec - exiting 03:57:27 (2244): No heartbeat from core client for 30 sec - exiting 03:57:28 (2244): No heartbeat from core client for 30 sec - exiting 03:57:29 (2244): No heartbeat from core client for 30 sec - exiting 03:57:30 (2244): No heartbeat from core client for 30 sec - exiting 07:38:04 (2244): No heartbeat from core client for 30 sec - exiting 07:38:05 (2244): No heartbeat from core client for 30 sec - exiting 07:38:06 (2244): No heartbeat from core client for 30 sec - exiting 07:38:07 (2244): No heartbeat from core client for 30 sec - exiting 07:38:08 (2244): No heartbeat from core client for 30 sec - exiting 07:38:09 (2244): No heartbeat from core client for 30 sec - exiting 07:38:40 (2244): No heartbeat from core client for 30 sec - exiting 07:38:41 (2244): No heartbeat from core client for 30 sec - exiting 07:38:42 (2244): No heartbeat from core client for 30 sec - exiting 07:38:43 (2244): No heartbeat from core client for 30 sec - exiting 07:38:44 (2244): No heartbeat from core client for 30 sec - exiting 07:38:45 (2244): No heartbeat from core client for 30 sec - exiting 07:38:46 (2244): No heartbeat from core client for 30 sec - exiting 07:38:47 (2244): No heartbeat from core client for 30 sec - exiting 07:38:48 (2244): No heartbeat from core client for 30 sec - exiting 07:38:49 (2244): No heartbeat from core client for 30 sec - exiting 02:39:50 (2244): No heartbeat from core client for 30 sec - exiting 02:39:52 (2244): No heartbeat from core client for 30 sec - exiting 02:39:53 (2244): No heartbeat from core client for 30 sec - exiting 02:39:54 (2244): No heartbeat from core client for 30 sec - exiting 02:39:55 (2244): No heartbeat from core client for 30 sec - exiting 02:39:56 (2244): No heartbeat from core client for 30 sec - exiting 07:04:16 (2244): No heartbeat from core client for 30 sec - exiting 07:04:17 (2244): No heartbeat from core client for 30 sec - exiting 07:04:18 (2244): No heartbeat from core client for 30 sec - exiting 07:04:19 (2244): No heartbeat from core client for 30 sec - exiting 07:04:20 (2244): No heartbeat from core client for 30 sec - exiting 07:04:21 (2244): No heartbeat from core client for 30 sec - exiting 07:04:22 (2244): No heartbeat from core client for 30 sec - exiting 07:04:23 (2244): No heartbeat from core client for 30 sec - exiting 07:04:24 (2244): No heartbeat from core client for 30 sec - exiting 07:04:25 (2244): No heartbeat from core client for 30 sec - exiting 07:04:26 (2244): No heartbeat from core client for 30 sec - exiting 07:04:27 (2244): No heartbeat from core client for 30 sec - exiting 07:04:28 (2244): No heartbeat from core client for 30 sec - exiting 07:04:29 (2244): No heartbeat from core client for 30 sec - exiting 07:04:30 (2244): No heartbeat from core client for 30 sec - exiting 07:04:31 (2244): No heartbeat from core client for 30 sec - exiting 07:54:38 (2244): No heartbeat from core client for 30 sec - exiting 13:48:57 (2244): No heartbeat from core client for 30 sec - exiting 14:05:14 (2244): No heartbeat from core client for 30 sec - exiting 14:05:15 (2244): No heartbeat from core client for 30 sec - exiting 14:05:16 (2244): No heartbeat from core client for 30 sec - exiting 14:05:17 (2244): No heartbeat from core client for 30 sec - exiting 14:05:18 (2244): No heartbeat from core client for 30 sec - exiting 14:05:19 (2244): No heartbeat from core client for 30 sec - exiting 14:05:20 (2244): No heartbeat from core client for 30 sec - exiting 14:05:21 (2244): No heartbeat from core client for 30 sec - exiting 14:05:22 (2244): No heartbeat from core client for 30 sec - exiting 14:05:23 (2244): No heartbeat from core client for 30 sec - exiting 14:05:24 (2244): No heartbeat from core client for 30 sec - exiting 05:05:47 (2244): No heartbeat from core client for 30 sec - exiting 05:05:48 (2244): No heartbeat from core client for 30 sec - exiting 05:05:49 (2244): No heartbeat from core client for 30 sec - exiting 05:05:50 (2244): No heartbeat from core client for 30 sec - exiting 05:05:51 (2244): No heartbeat from core client for 30 sec - exiting 05:05:52 (2244): No heartbeat from core client for 30 sec - exiting 05:12:33 (2244): No heartbeat from core client for 30 sec - exiting 05:12:34 (2244): No heartbeat from core client for 30 sec - exiting 05:12:35 (2244): No heartbeat from core client for 30 sec - exiting 05:12:36 (2244): No heartbeat from core client for 30 sec - exiting 05:12:37 (2244): No heartbeat from core client for 30 sec - exiting * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x097bd888 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 04:31:05 (10352): No heartbeat from core client for 30 sec - exiting 06:03:26 (10352): No heartbeat from core client for 30 sec - exiting 06:03:27 (10352): No heartbeat from core client for 30 sec - exiting 06:03:28 (10352): No heartbeat from core client for 30 sec - exiting 06:05:00 (10352): No heartbeat from core client for 30 sec - exiting 06:51:28 (10352): No heartbeat from core client for 30 sec - exiting 06:51:29 (10352): No heartbeat from core client for 30 sec - exiting 06:51:30 (10352): No heartbeat from core client for 30 sec - exiting 06:51:31 (10352): No heartbeat from core client for 30 sec - exiting 06:51:32 (10352): No heartbeat from core client for 30 sec - exiting 06:51:33 (10352): No heartbeat from core client for 30 sec - exiting 06:51:34 (10352): No heartbeat from core client for 30 sec - exiting 06:51:35 (10352): No heartbeat from core client for 30 sec - exiting 06:51:36 (10352): No heartbeat from core client for 30 sec - exiting 06:51:37 (10352): No heartbeat from core client for 30 sec - exiting 06:52:39 (10352): No heartbeat from core client for 30 sec - exiting 08:03:03 (10352): No heartbeat from core client for 30 sec - exiting 09:35:03 (10352): No heartbeat from core client for 30 sec - exiting 09:35:04 (10352): No heartbeat from core client for 30 sec - exiting 09:35:05 (10352): No heartbeat from core client for 30 sec - exiting 09:35:06 (10352): No heartbeat from core client for 30 sec - exiting 09:35:07 (10352): No heartbeat from core client for 30 sec - exiting 09:35:08 (10352): No heartbeat from core client for 30 sec - exiting 09:35:09 (10352): No heartbeat from core client for 30 sec - exiting 09:35:10 (10352): No heartbeat from core client for 30 sec - exiting 09:35:11 (10352): No heartbeat from core client for 30 sec - exiting 09:35:12 (10352): No heartbeat from core client for 30 sec - exiting 09:35:43 (10352): No heartbeat from core client for 30 sec - exiting 09:35:44 (10352): No heartbeat from core client for 30 sec - exiting 09:35:45 (10352): No heartbeat from core client for 30 sec - exiting 09:35:46 (10352): No heartbeat from core client for 30 sec - exiting 09:35:47 (10352): No heartbeat from core client for 30 sec - exiting 09:52:11 (10352): No heartbeat from core client for 30 sec - exiting 09:52:12 (10352): No heartbeat from core client for 30 sec - exiting 09:52:13 (10352): No heartbeat from core client for 30 sec - exiting 09:52:14 (10352): No heartbeat from core client for 30 sec - exiting 09:52:15 (10352): No heartbeat from core client for 30 sec - exiting 09:52:16 (10352): No heartbeat from core client for 30 sec - exiting 09:52:17 (10352): No heartbeat from core client for 30 sec - exiting 09:52:18 (10352): No heartbeat from core client for 30 sec - exiting 09:52:19 (10352): No heartbeat from core client for 30 sec - exiting 09:52:20 (10352): No heartbeat from core client for 30 sec - exiting 09:52:51 (10352): No heartbeat from core client for 30 sec - exiting 09:54:26 (10352): No heartbeat from core client for 30 sec - exiting 09:54:27 (10352): No heartbeat from core client for 30 sec - exiting 09:54:28 (10352): No heartbeat from core client for 30 sec - exiting 09:54:29 (10352): No heartbeat from core client for 30 sec - exiting 09:54:30 (10352): No heartbeat from core client for 30 sec - exiting 09:54:31 (10352): No heartbeat from core client for 30 sec - exiting 09:54:32 (10352): No heartbeat from core client for 30 sec - exiting 09:54:33 (10352): No heartbeat from core client for 30 sec - exiting 09:54:34 (10352): No heartbeat from core client for 30 sec - exiting 09:54:35 (10352): No heartbeat from core client for 30 sec - exiting 09:55:31 (10352): No heartbeat from core client for 30 sec - exiting 09:55:32 (10352): No heartbeat from core client for 30 sec - exiting 09:55:33 (10352): No heartbeat from core client for 30 sec - exiting 09:55:34 (10352): No heartbeat from core client for 30 sec - exiting 09:55:35 (10352): No heartbeat from core client for 30 sec - exiting 09:55:36 (10352): No heartbeat from core client for 30 sec - exiting 09:55:37 (10352): No heartbeat from core client for 30 sec - exiting 09:55:38 (10352): No heartbeat from core client for 30 sec - exiting 09:55:39 (10352): No heartbeat from core client for 30 sec - exiting 09:55:40 (10352): No heartbeat from core client for 30 sec - exiting 11:29:41 (10352): No heartbeat from core client for 30 sec - exiting 02:05:46 (10352): No heartbeat from core client for 30 sec - exiting 02:05:47 (10352): No heartbeat from core client for 30 sec - exiting 02:05:48 (10352): No heartbeat from core client for 30 sec - exiting 02:05:49 (10352): No heartbeat from core client for 30 sec - exiting 02:05:50 (10352): No heartbeat from core client for 30 sec - exiting 03:19:35 (10352): No heartbeat from core client for 30 sec - exiting 03:19:36 (10352): No heartbeat from core client for 30 sec - exiting 03:19:37 (10352): No heartbeat from core client for 30 sec - exiting 03:19:38 (10352): No heartbeat from core client for 30 sec - exiting 03:19:39 (10352): No heartbeat from core client for 30 sec - exiting 03:19:40 (10352): No heartbeat from core client for 30 sec - exiting 03:19:41 (10352): No heartbeat from core client for 30 sec - exiting 03:19:42 (10352): No heartbeat from core client for 30 sec - exiting 03:19:43 (10352): No heartbeat from core client for 30 sec - exiting 03:19:44 (10352): No heartbeat from core client for 30 sec - exiting 03:27:44 (10352): No heartbeat from core client for 30 sec - exiting 03:27:46 (10352): No heartbeat from core client for 30 sec - exiting 03:27:47 (10352): No heartbeat from core client for 30 sec - exiting 03:27:48 (10352): No heartbeat from core client for 30 sec - exiting 03:27:49 (10352): No heartbeat from core client for 30 sec - exiting 03:27:50 (10352): No heartbeat from core client for 30 sec - exiting 03:27:51 (10352): No heartbeat from core client for 30 sec - exiting 03:27:52 (10352): No heartbeat from core client for 30 sec - exiting 03:27:53 (10352): No heartbeat from core client for 30 sec - exiting 03:27:54 (10352): No heartbeat from core client for 30 sec - exiting 04:37:34 (10352): No heartbeat from core client for 30 sec - exiting 04:37:35 (10352): No heartbeat from core client for 30 sec - exiting 04:37:36 (10352): No heartbeat from core client for 30 sec - exiting 09:32:03 (10352): No heartbeat from core client for 30 sec - exiting 09:32:04 (10352): No heartbeat from core client for 30 sec - exiting 15:09:38 (10352): No heartbeat from core client for 30 sec - exiting 15:09:39 (10352): No heartbeat from core client for 30 sec - exiting 15:12:52 (10352): No heartbeat from core client for 30 sec - exiting 15:12:53 (10352): No heartbeat from core client for 30 sec - exiting 15:12:54 (10352): No heartbeat from core client for 30 sec - exiting 15:12:55 (10352): No heartbeat from core client for 30 sec - exiting 15:12:56 (10352): No heartbeat from core client for 30 sec - exiting 15:12:57 (10352): No heartbeat from core client for 30 sec - exiting 15:15:50 (10352): No heartbeat from core client for 30 sec - exiting 15:15:51 (10352): No heartbeat from core client for 30 sec - exiting 15:15:52 (10352): No heartbeat from core client for 30 sec - exiting 15:15:53 (10352): No heartbeat from core client for 30 sec - exiting 15:15:54 (10352): No heartbeat from core client for 30 sec - exiting 15:15:55 (10352): No heartbeat from core client for 30 sec - exiting 15:15:56 (10352): No heartbeat from core client for 30 sec - exiting 15:15:57 (10352): No heartbeat from core client for 30 sec - exiting 15:15:58 (10352): No heartbeat from core client for 30 sec - exiting 15:15:59 (10352): No heartbeat from core client for 30 sec - exiting 15:16:00 (10352): No heartbeat from core client for 30 sec - exiting 15:16:01 (10352): No heartbeat from core client for 30 sec - exiting 15:16:02 (10352): No heartbeat from core client for 30 sec - exiting 15:16:03 (10352): No heartbeat from core client for 30 sec - exiting 15:16:04 (10352): No heartbeat from core client for 30 sec - exiting 15:16:05 (10352): No heartbeat from core client for 30 sec - exiting 15:16:06 (10352): No heartbeat from core client for 30 sec - exiting 15:16:07 (10352): No heartbeat from core client for 30 sec - exiting 15:16:08 (10352): No heartbeat from core client for 30 sec - exiting 15:16:09 (10352): No heartbeat from core client for 30 sec - exiting 15:16:10 (10352): No heartbeat from core client for 30 sec - exiting 15:16:11 (10352): No heartbeat from core client for 30 sec - exiting 15:16:12 (10352): No heartbeat from core client for 30 sec - exiting 15:16:13 (10352): No heartbeat from core client for 30 sec - exiting 15:16:14 (10352): No heartbeat from core client for 30 sec - exiting 15:16:15 (10352): No heartbeat from core client for 30 sec - exiting 15:16:16 (10352): No heartbeat from core client for 30 sec - exiting * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x096f5888 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 07:04:54 (2068): No heartbeat from core client for 30 sec - exiting 07:04:55 (2068): No heartbeat from core client for 30 sec - exiting 07:04:56 (2068): No heartbeat from core client for 30 sec - exiting 07:04:57 (2068): No heartbeat from core client for 30 sec - exiting 07:04:58 (2068): No heartbeat from core client for 30 sec - exiting 07:04:59 (2068): No heartbeat from core client for 30 sec - exiting 07:05:00 (2068): No heartbeat from core client for 30 sec - exiting 07:05:01 (2068): No heartbeat from core client for 30 sec - exiting 07:05:02 (2068): No heartbeat from core client for 30 sec - exiting 07:05:03 (2068): No heartbeat from core client for 30 sec - exiting 07:05:04 (2068): No heartbeat from core client for 30 sec - exiting 07:05:37 (2068): No heartbeat from core client for 30 sec - exiting 07:05:38 (2068): No heartbeat from core client for 30 sec - exiting 07:05:39 (2068): No heartbeat from core client for 30 sec - exiting 07:05:40 (2068): No heartbeat from core client for 30 sec - exiting 07:05:41 (2068): No heartbeat from core client for 30 sec - exiting 07:05:42 (2068): No heartbeat from core client for 30 sec - exiting 07:05:43 (2068): No heartbeat from core client for 30 sec - exiting 07:05:44 (2068): No heartbeat from core client for 30 sec - exiting 07:05:45 (2068): No heartbeat from core client for 30 sec - exiting 07:05:46 (2068): No heartbeat from core client for 30 sec - exiting 07:05:47 (2068): No heartbeat from core client for 30 sec - exiting 07:06:18 (2068): No heartbeat from core client for 30 sec - exiting 07:06:19 (2068): No heartbeat from core client for 30 sec - exiting 07:06:20 (2068): No heartbeat from core client for 30 sec - exiting 07:06:21 (2068): No heartbeat from core client for 30 sec - exiting 07:06:22 (2068): No heartbeat from core client for 30 sec - exiting 03:19:56 (2068): No heartbeat from core client for 30 sec - exiting 03:19:57 (2068): No heartbeat from core client for 30 sec - exiting 03:19:58 (2068): No heartbeat from core client for 30 sec - exiting 03:19:59 (2068): No heartbeat from core client for 30 sec - exiting 03:20:00 (2068): No heartbeat from core client for 30 sec - exiting 03:20:01 (2068): No heartbeat from core client for 30 sec - exiting 03:20:58 (2068): No heartbeat from core client for 30 sec - exiting 03:20:59 (2068): No heartbeat from core client for 30 sec - exiting 03:21:00 (2068): No heartbeat from core client for 30 sec - exiting 03:21:01 (2068): No heartbeat from core client for 30 sec - exiting 03:21:02 (2068): No heartbeat from core client for 30 sec - exiting 03:21:03 (2068): No heartbeat from core client for 30 sec - exiting 03:21:04 (2068): No heartbeat from core client for 30 sec - exiting 03:21:05 (2068): No heartbeat from core client for 30 sec - exiting 14:49:51 (2068): No heartbeat from core client for 30 sec - exiting 02:26:24 (2068): No heartbeat from core client for 30 sec - exiting 02:26:25 (2068): No heartbeat from core client for 30 sec - exiting 02:26:26 (2068): No heartbeat from core client for 30 sec - exiting 02:26:27 (2068): No heartbeat from core client for 30 sec - exiting 02:26:28 (2068): No heartbeat from core client for 30 sec - exiting 02:26:29 (2068): No heartbeat from core client for 30 sec - exiting 02:26:30 (2068): No heartbeat from core client for 30 sec - exiting 02:26:31 (2068): No heartbeat from core client for 30 sec - exiting 02:26:32 (2068): No heartbeat from core client for 30 sec - exiting 02:26:33 (2068): No heartbeat from core client for 30 sec - exiting 02:26:34 (2068): No heartbeat from core client for 30 sec - exiting 04:07:15 (2068): No heartbeat from core client for 30 sec - exiting 04:07:16 (2068): No heartbeat from core client for 30 sec - exiting 04:07:17 (2068): No heartbeat from core client for 30 sec - exiting 04:07:18 (2068): No heartbeat from core client for 30 sec - exiting 04:07:19 (2068): No heartbeat from core client for 30 sec - exiting 04:07:20 (2068): No heartbeat from core client for 30 sec - exiting 03:25:35 (2068): No heartbeat from core client for 30 sec - exiting 03:25:37 (2068): No heartbeat from core client for 30 sec - exiting 03:25:38 (2068): No heartbeat from core client for 30 sec - exiting 03:25:39 (2068): No heartbeat from core client for 30 sec - exiting 03:25:40 (2068): No heartbeat from core client for 30 sec - exiting 03:25:41 (2068): No heartbeat from core client for 30 sec - exiting 07:33:39 (2068): No heartbeat from core client for 30 sec - exiting 10:40:07 (2068): No heartbeat from core client for 30 sec - exiting 10:40:08 (2068): No heartbeat from core client for 30 sec - exiting 10:40:09 (2068): No heartbeat from core client for 30 sec - exiting 10:40:10 (2068): No heartbeat from core client for 30 sec - exiting 10:40:11 (2068): No heartbeat from core client for 30 sec - exiting 10:40:12 (2068): No heartbeat from core client for 30 sec - exiting 11:47:31 (2068): No heartbeat from core client for 30 sec - exiting 11:02:52 (2068): No heartbeat from core client for 30 sec - exiting 11:02:53 (2068): No heartbeat from core client for 30 sec - exiting 11:02:54 (2068): No heartbeat from core client for 30 sec - exiting 11:02:55 (2068): No heartbeat from core client for 30 sec - exiting 11:02:56 (2068): No heartbeat from core client for 30 sec - exiting 11:02:57 (2068): No heartbeat from core client for 30 sec - exiting 11:02:58 (2068): No heartbeat from core client for 30 sec - exiting 11:02:59 (2068): No heartbeat from core client for 30 sec - exiting 11:03:00 (2068): No heartbeat from core client for 30 sec - exiting 11:03:01 (2068): No heartbeat from core client for 30 sec - exiting 11:03:02 (2068): No heartbeat from core client for 30 sec - exiting 11:52:34 (2068): No heartbeat from core client for 30 sec - exiting 11:52:36 (2068): No heartbeat from core client for 30 sec - exiting 11:52:37 (2068): No heartbeat from core client for 30 sec - exiting 11:52:38 (2068): No heartbeat from core client for 30 sec - exiting 11:52:39 (2068): No heartbeat from core client for 30 sec - exiting 11:58:25 (2068): No heartbeat from core client for 30 sec - exiting 12:38:41 (2068): No heartbeat from core client for 30 sec - exiting 22:32:37 (2068): No heartbeat from core client for 30 sec - exiting 22:32:38 (2068): No heartbeat from core client for 30 sec - exiting 22:32:39 (2068): No heartbeat from core client for 30 sec - exiting 22:32:40 (2068): No heartbeat from core client for 30 sec - exiting 22:32:41 (2068): No heartbeat from core client for 30 sec - exiting 22:32:42 (2068): No heartbeat from core client for 30 sec - exiting 22:32:43 (2068): No heartbeat from core client for 30 sec - exiting 22:32:44 (2068): No heartbeat from core client for 30 sec - exiting 22:32:45 (2068): No heartbeat from core client for 30 sec - exiting 22:32:46 (2068): No heartbeat from core client for 30 sec - exiting 22:32:47 (2068): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x099b3888 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 01:04:34 (26424): No heartbeat from core client for 30 sec - exiting 01:04:35 (26424): No heartbeat from core client for 30 sec - exiting 01:04:36 (26424): No heartbeat from core client for 30 sec - exiting 01:04:37 (26424): No heartbeat from core client for 30 sec - exiting 01:04:38 (26424): No heartbeat from core client for 30 sec - exiting 01:04:39 (26424): No heartbeat from core client for 30 sec - exiting 01:05:13 (26424): No heartbeat from core client for 30 sec - exiting 22:30:33 (26424): No heartbeat from core client for 30 sec - exiting 22:30:34 (26424): No heartbeat from core client for 30 sec - exiting 22:30:35 (26424): No heartbeat from core client for 30 sec - exiting 22:30:36 (26424): No heartbeat from core client for 30 sec - exiting 22:30:37 (26424): No heartbeat from core client for 30 sec - exiting 22:30:38 (26424): No heartbeat from core client for 30 sec - exiting 22:30:39 (26424): No heartbeat from core client for 30 sec - exiting 22:30:40 (26424): No heartbeat from core client for 30 sec - exiting 22:30:41 (26424): No heartbeat from core client for 30 sec - exiting 22:30:42 (26424): No heartbeat from core client for 30 sec - exiting 22:30:43 (26424): No heartbeat from core client for 30 sec - exiting 04:11:09 (26424): No heartbeat from core client for 30 sec - exiting 04:11:10 (26424): No heartbeat from core client for 30 sec - exiting 04:11:11 (26424): No heartbeat from core client for 30 sec - exiting 04:11:12 (26424): No heartbeat from core client for 30 sec - exiting 04:11:13 (26424): No heartbeat from core client for 30 sec - exiting 15:26:43 (26424): No heartbeat from core client for 30 sec - exiting 15:26:44 (26424): No heartbeat from core client for 30 sec - exiting 15:26:45 (26424): No heartbeat from core client for 30 sec - exiting 15:26:46 (26424): No heartbeat from core client for 30 sec - exiting 15:26:47 (26424): No heartbeat from core client for 30 sec - exiting 15:26:48 (26424): No heartbeat from core client for 30 sec - exiting 15:26:49 (26424): No heartbeat from core client for 30 sec - exiting 15:26:50 (26424): No heartbeat from core client for 30 sec - exiting 15:26:51 (26424): No heartbeat from core client for 30 sec - exiting 15:26:52 (26424): No heartbeat from core client for 30 sec - exiting 15:27:23 (26424): No heartbeat from core client for 30 sec - exiting 15:27:24 (26424): No heartbeat from core client for 30 sec - exiting 15:27:25 (26424): No heartbeat from core client for 30 sec - exiting 15:27:26 (26424): No heartbeat from core client for 30 sec - exiting 15:27:27 (26424): No heartbeat from core client for 30 sec - exiting 15:27:28 (26424): No heartbeat from core client for 30 sec - exiting 15:27:29 (26424): No heartbeat from core client for 30 sec - exiting 15:27:30 (26424): No heartbeat from core client for 30 sec - exiting 15:27:31 (26424): No heartbeat from core client for 30 sec - exiting 15:27:32 (26424): No heartbeat from core client for 30 sec - exiting 08:35:50 (26424): No heartbeat from core client for 30 sec - exiting 02:27:27 (26424): No heartbeat from core client for 30 sec - exiting 04:04:40 (26424): No heartbeat from core client for 30 sec - exiting 04:04:41 (26424): No heartbeat from core client for 30 sec - exiting 04:04:42 (26424): No heartbeat from core client for 30 sec - exiting 04:04:43 (26424): No heartbeat from core client for 30 sec - exiting 04:04:44 (26424): No heartbeat from core client for 30 sec - exiting 04:04:45 (26424): No heartbeat from core client for 30 sec - exiting 04:04:46 (26424): No heartbeat from core client for 30 sec - exiting 04:04:47 (26424): No heartbeat from core client for 30 sec - exiting 04:04:48 (26424): No heartbeat from core client for 30 sec - exiting 04:04:49 (26424): No heartbeat from core client for 30 sec - exiting 04:04:50 (26424): No heartbeat from core client for 30 sec - exiting 04:04:51 (26424): No heartbeat from core client for 30 sec - exiting 04:04:52 (26424): No heartbeat from core client for 30 sec - exiting 04:04:53 (26424): No heartbeat from core client for 30 sec - exiting 04:04:54 (26424): No heartbeat from core client for 30 sec - exiting 04:04:55 (26424): No heartbeat from core client for 30 sec - exiting 04:04:56 (26424): No heartbeat from core client for 30 sec - exiting 04:04:57 (26424): No heartbeat from core client for 30 sec - exiting 04:04:58 (26424): No heartbeat from core client for 30 sec - exiting 04:04:59 (26424): No heartbeat from core client for 30 sec - exiting 04:05:00 (26424): No heartbeat from core client for 30 sec - exiting 04:05:01 (26424): No heartbeat from core client for 30 sec - exiting 04:05:02 (26424): No heartbeat from core client for 30 sec - exiting 04:05:03 (26424): No heartbeat from core client for 30 sec - exiting 04:05:04 (26424): No heartbeat from core client for 30 sec - exiting 04:05:05 (26424): No heartbeat from core client for 30 sec - exiting 04:05:06 (26424): No heartbeat from core client for 30 sec - exiting 04:05:07 (26424): No heartbeat from core client for 30 sec - exiting 04:05:08 (26424): No heartbeat from core client for 30 sec - exiting 04:05:09 (26424): No heartbeat from core client for 30 sec - exiting 04:05:10 (26424): No heartbeat from core client for 30 sec - exiting 04:05:11 (26424): No heartbeat from core client for 30 sec - exiting 07:43:57 (26424): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Dec 2012 00:56:53	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	259,200	346,096	1.3352
04 Dec 2012 23:52:30	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	233,280	311,417	1.3349
04 Dec 2012 06:16:38	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	207,360	276,849	1.3351
04 Dec 2012 00:35:21	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	181,440	242,983	1.3392
30 Nov 2012 18:33:48	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	155,520	208,459	1.3404
29 Nov 2012 04:16:37	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	129,600	174,716	1.3481
26 Nov 2012 20:14:29	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	103,680	141,399	1.3638
26 Nov 2012 01:08:29	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	77,760	107,833	1.3867
25 Nov 2012 17:48:38	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	51,840	72,614	1.4007
24 Nov 2012 13:09:23	1219037	15454043	hadcm3n_zhgb_1880_40_008251308_2	25,920	36,967	1.4262