climateprediction.net home page
Task 15451191

Task 15451191

Name hadcm3n_zmh3_1880_40_008250218_1
Workunit 8405342
Created 22 Nov 2012, 4:08:03 UTC
Sent 22 Nov 2012, 4:08:20 UTC
Report deadline 21 Feb 2013, 11:35:31 UTC
Received 21 Nov 2013, 16:30:33 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1242159
Run time 8 days 11 hours 53 min 31 sec
CPU time 8 days 10 hours 27 min 17 sec
Validate state Invalid
Credit 3,421.44
Device peak FLOPS 1.99 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
r 30 sec - exiting
19:32:33 (32519): No heartbeat from core client for 30 sec - exiting
19:33:14 (32519): No heartbeat from core client for 30 sec - exiting
19:33:15 (32519): No heartbeat from core client for 30 sec - exiting
19:33:16 (32519): No heartbeat from core client for 30 sec - exiting
19:33:17 (32519): No heartbeat from core client for 30 sec - exiting
19:33:18 (32519): No heartbeat from core client for 30 sec - exiting
19:33:19 (32519): No heartbeat from core client for 30 sec - exiting
19:33:20 (32519): No heartbeat from core client for 30 sec - exiting
19:33:21 (32519): No heartbeat from core client for 30 sec - exiting
19:33:22 (32519): No heartbeat from core client for 30 sec - exiting
19:33:23 (32519): No heartbeat from core client for 30 sec - exiting
19:33:24 (32519): No heartbeat from core client for 30 sec - exiting
19:33:25 (32519): No heartbeat from core client for 30 sec - exiting
19:33:26 (32519): No heartbeat from core client for 30 sec - exiting
19:33:27 (32519): No heartbeat from core client for 30 sec - exiting
19:33:28 (32519): No heartbeat from core client for 30 sec - exiting
19:33:29 (32519): No heartbeat from core client for 30 sec - exiting
19:33:30 (32519): No heartbeat from core client for 30 sec - exiting
19:33:31 (32519): No heartbeat from core client for 30 sec - exiting
19:33:32 (32519): No heartbeat from core client for 30 sec - exiting
19:33:33 (32519): No heartbeat from core client for 30 sec - exiting
19:33:34 (32519): No heartbeat from core client for 30 sec - exiting
19:33:35 (32519): No heartbeat from core client for 30 sec - exiting
19:33:36 (32519): No heartbeat from core client for 30 sec - exiting
19:33:37 (32519): No heartbeat from core client for 30 sec - exiting
19:33:38 (32519): No heartbeat from core client for 30 sec - exiting
19:33:39 (32519): No heartbeat from core client for 30 sec - exiting
19:33:40 (32519): No heartbeat from core client for 30 sec - exiting
19:33:41 (32519): No heartbeat from core client for 30 sec - exiting
19:33:42 (32519): No heartbeat from core client for 30 sec - exiting
19:33:43 (32519): No heartbeat from core client for 30 sec - exiting
19:33:44 (32519): No heartbeat from core client for 30 sec - exiting
08:47:01 (32519): No heartbeat from core client for 30 sec - exiting
22:15:57 (32519): No heartbeat from core client for 30 sec - exiting
22:16:38 (32519): No heartbeat from core client for 30 sec - exiting
22:16:39 (32519): No heartbeat from core client for 30 sec - exiting
22:16:40 (32519): No heartbeat from core client for 30 sec - exiting
22:16:41 (32519): No heartbeat from core client for 30 sec - exiting
22:16:42 (32519): No heartbeat from core client for 30 sec - exiting
22:16:43 (32519): No heartbeat from core client for 30 sec - exiting
22:16:44 (32519): No heartbeat from core client for 30 sec - exiting
22:16:45 (32519): No heartbeat from core client for 30 sec - exiting
22:16:46 (32519): No heartbeat from core client for 30 sec - exiting
22:16:47 (32519): No heartbeat from core client for 30 sec - exiting
22:16:48 (32519): No heartbeat from core client for 30 sec - exiting
22:16:49 (32519): No heartbeat from core client for 30 sec - exiting
22:16:50 (32519): No heartbeat from core client for 30 sec - exiting
22:16:51 (32519): No heartbeat from core client for 30 sec - exiting
22:16:52 (32519): No heartbeat from core client for 30 sec - exiting
22:16:53 (32519): No heartbeat from core client for 30 sec - exiting
22:16:54 (32519): No heartbeat from core client for 30 sec - exiting
22:16:55 (32519): No heartbeat from core client for 30 sec - exiting
22:16:56 (32519): No heartbeat from core client for 30 sec - exiting
22:16:57 (32519): No heartbeat from core client for 30 sec - exiting
22:16:58 (32519): No heartbeat from core client for 30 sec - exiting
22:16:59 (32519): No heartbeat from core client for 30 sec - exiting
22:17:00 (32519): No heartbeat from core client for 30 sec - exiting
22:17:01 (32519): No heartbeat from core client for 30 sec - exiting
22:17:02 (32519): No heartbeat from core client for 30 sec - exiting
22:17:03 (32519): No heartbeat from core client for 30 sec - exiting
22:17:04 (32519): No heartbeat from core client for 30 sec - exiting
22:17:05 (32519): No heartbeat from core client for 30 sec - exiting
22:17:06 (32519): No heartbeat from core client for 30 sec - exiting
22:17:07 (32519): No heartbeat from core client for 30 sec - exiting
22:17:08 (32519): No heartbeat from core client for 30 sec - exiting
22:17:09 (32519): No heartbeat from core client for 30 sec - exiting
22:17:10 (32519): No heartbeat from core client for 30 sec - exiting
22:17:11 (32519): No heartbeat from core client for 30 sec - exiting
22:17:12 (32519): No heartbeat from core client for 30 sec - exiting
22:17:13 (32519): No heartbeat from core client for 30 sec - exiting
22:17:14 (32519): No heartbeat from core client for 30 sec - exiting
22:17:15 (32519): No heartbeat from core client for 30 sec - exiting
22:17:16 (32519): No heartbeat from core client for 30 sec - exiting
22:17:17 (32519): No heartbeat from core client for 30 sec - exiting
22:17:18 (32519): No heartbeat from core client for 30 sec - exiting
22:17:19 (32519): No heartbeat from core client for 30 sec - exiting
22:17:20 (32519): No heartbeat from core client for 30 sec - exiting
22:17:21 (32519): No heartbeat from core client for 30 sec - exiting
22:17:22 (32519): No heartbeat from core client for 30 sec - exiting
22:18:00 (32519): No heartbeat from core client for 30 sec - exiting
22:18:01 (32519): No heartbeat from core client for 30 sec - exiting
22:18:02 (32519): No heartbeat from core client for 30 sec - exiting
22:18:03 (32519): No heartbeat from core client for 30 sec - exiting
22:18:04 (32519): No heartbeat from core client for 30 sec - exiting
22:18:05 (32519): No heartbeat from core client for 30 sec - exiting
22:18:06 (32519): No heartbeat from core client for 30 sec - exiting
22:18:07 (32519): No heartbeat from core client for 30 sec - exiting
22:18:08 (32519): No heartbeat from core client for 30 sec - exiting
22:18:09 (32519): No heartbeat from core client for 30 sec - exiting
22:18:10 (32519): No heartbeat from core client for 30 sec - exiting
22:18:11 (32519): No heartbeat from core client for 30 sec - exiting
22:18:12 (32519): No heartbeat from core client for 30 sec - exiting
22:18:13 (32519): No heartbeat from core client for 30 sec - exiting
22:18:14 (32519): No heartbeat from core client for 30 sec - exiting
22:18:15 (32519): No heartbeat from core client for 30 sec - exiting
22:18:16 (32519): No heartbeat from core client for 30 sec - exiting
22:18:17 (32519): No heartbeat from core client for 30 sec - exiting
07:57:58 (32519): No heartbeat from core client for 30 sec - exiting
07:58:02 (32519): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0996f810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
22:09:16 (2606): No heartbeat from core client for 30 sec - exiting
09:41:47 (2606): No heartbeat from core client for 30 sec - exiting
10:05:58 (2606): No heartbeat from core client for 30 sec - exiting
10:06:48 (2606): No heartbeat from core client for 30 sec - exiting
08:00:31 (2606): No heartbeat from core client for 30 sec - exiting
18:48:25 (2606): No heartbeat from core client for 30 sec - exiting
18:49:23 (2606): No heartbeat from core client for 30 sec - exiting
18:49:24 (2606): No heartbeat from core client for 30 sec - exiting
18:49:25 (2606): No heartbeat from core client for 30 sec - exiting
18:49:26 (2606): No heartbeat from core client for 30 sec - exiting
18:49:27 (2606): No heartbeat from core client for 30 sec - exiting
07:39:22 (2606): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09b05810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0988d810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x095da810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
08:14:45 (8433): No heartbeat from core client for 30 sec - exiting
09:41:00 (8433): No heartbeat from core client for 30 sec - exiting
09:41:39 (8433): No heartbeat from core client for 30 sec - exiting
09:41:40 (8433): No heartbeat from core client for 30 sec - exiting
01:07:02 (8433): No heartbeat from core client for 30 sec - exiting
01:08:16 (8433): No heartbeat from core client for 30 sec - exiting
01:08:17 (8433): No heartbeat from core client for 30 sec - exiting
01:08:18 (8433): No heartbeat from core client for 30 sec - exiting
01:08:19 (8433): No heartbeat from core client for 30 sec - exiting
01:08:20 (8433): No heartbeat from core client for 30 sec - exiting
01:08:21 (8433): No heartbeat from core client for 30 sec - exiting
01:08:22 (8433): No heartbeat from core client for 30 sec - exiting
01:08:23 (8433): No heartbeat from core client for 30 sec - exiting
01:08:28 (8433): No heartbeat from core client for 30 sec - exiting
12:35:18 (8433): No heartbeat from core client for 30 sec - exiting
12:35:29 (8433): No heartbeat from core client for 30 sec - exiting
12:35:30 (8433): No heartbeat from core client for 30 sec - exiting
12:35:31 (8433): No heartbeat from core client for 30 sec - exiting
12:35:32 (8433): No heartbeat from core client for 30 sec - exiting
12:35:33 (8433): No heartbeat from core client for 30 sec - exiting
12:35:34 (8433): No heartbeat from core client for 30 sec - exiting
12:35:35 (8433): No heartbeat from core client for 30 sec - exiting
12:35:36 (8433): No heartbeat from core client for 30 sec - exiting
12:35:37 (8433): No heartbeat from core client for 30 sec - exiting
12:35:38 (8433): No heartbeat from core client for 30 sec - exiting
12:35:39 (8433): No heartbeat from core client for 30 sec - exiting
12:35:40 (8433): No heartbeat from core client for 30 sec - exiting
12:35:41 (8433): No heartbeat from core client for 30 sec - exiting
12:35:42 (8433): No heartbeat from core client for 30 sec - exiting
12:35:43 (8433): No heartbeat from core client for 30 sec - exiting
12:35:44 (8433): No heartbeat from core client for 30 sec - exiting
12:35:45 (8433): No heartbeat from core client for 30 sec - exiting
12:35:46 (8433): No heartbeat from core client for 30 sec - exiting
12:35:47 (8433): No heartbeat from core client for 30 sec - exiting
12:35:48 (8433): No heartbeat from core client for 30 sec - exiting
12:35:49 (8433): No heartbeat from core client for 30 sec - exiting
12:35:50 (8433): No heartbeat from core client for 30 sec - exiting
12:35:51 (8433): No heartbeat from core client for 30 sec - exiting
12:35:52 (8433): No heartbeat from core client for 30 sec - exiting
12:35:53 (8433): No heartbeat from core client for 30 sec - exiting
12:35:54 (8433): No heartbeat from core client for 30 sec - exiting
12:35:55 (8433): No heartbeat from core client for 30 sec - exiting
12:35:56 (8433): No heartbeat from core client for 30 sec - exiting
12:35:57 (8433): No heartbeat from core client for 30 sec - exiting
12:35:58 (8433): No heartbeat from core client for 30 sec - exiting
12:35:59 (8433): No heartbeat from core client for 30 sec - exiting
12:36:00 (8433): No heartbeat from core client for 30 sec - exiting
12:36:01 (8433): No heartbeat from core client for 30 sec - exiting
12:36:02 (8433): No heartbeat from core client for 30 sec - exiting
12:36:03 (8433): No heartbeat from core client for 30 sec - exiting
12:36:04 (8433): No heartbeat from core client for 30 sec - exiting
12:36:05 (8433): No heartbeat from core client for 30 sec - exiting
12:36:06 (8433): No heartbeat from core client for 30 sec - exiting
12:36:07 (8433): No heartbeat from core client for 30 sec - exiting
12:36:08 (8433): No heartbeat from core client for 30 sec - exiting
12:36:09 (8433): No heartbeat from core client for 30 sec - exiting
12:36:10 (8433): No heartbeat from core client for 30 sec - exiting
12:36:11 (8433): No heartbeat from core client for 30 sec - exiting
12:36:12 (8433): No heartbeat from core client for 30 sec - exiting
12:36:13 (8433): No heartbeat from core client for 30 sec - exiting
12:36:14 (8433): No heartbeat from core client for 30 sec - exiting
12:36:15 (8433): No heartbeat from core client for 30 sec - exiting
12:36:21 (8433): No heartbeat from core client for 30 sec - exiting
12:36:22 (8433): No heartbeat from core client for 30 sec - exiting
12:36:23 (8433): No heartbeat from core client for 30 sec - exiting
12:36:24 (8433): No heartbeat from core client for 30 sec - exiting
12:36:25 (8433): No heartbeat from core client for 30 sec - exiting
12:36:26 (8433): No heartbeat from core client for 30 sec - exiting
12:36:27 (8433): No heartbeat from core client for 30 sec - exiting
12:36:28 (8433): No heartbeat from core client for 30 sec - exiting
12:36:29 (8433): No heartbeat from core client for 30 sec - exiting
12:36:30 (8433): No heartbeat from core client for 30 sec - exiting
12:37:10 (8433): No heartbeat from core client for 30 sec - exiting
12:37:11 (8433): No heartbeat from core client for 30 sec - exiting
12:37:12 (8433): No heartbeat from core client for 30 sec - exiting
12:37:13 (8433): No heartbeat from core client for 30 sec - exiting
12:37:14 (8433): No heartbeat from core client for 30 sec - exiting
12:37:15 (8433): No heartbeat from core client for 30 sec - exiting
12:37:16 (8433): No heartbeat from core client for 30 sec - exiting
12:37:17 (8433): No heartbeat from core client for 30 sec - exiting
12:37:18 (8433): No heartbeat from core client for 30 sec - exiting
12:37:19 (8433): No heartbeat from core client for 30 sec - exiting
12:37:20 (8433): No heartbeat from core client for 30 sec - exiting
12:37:21 (8433): No heartbeat from core client for 30 sec - exiting
12:37:22 (8433): No heartbeat from core client for 30 sec - exiting
12:37:23 (8433): No heartbeat from core client for 30 sec - exiting
12:37:24 (8433): No heartbeat from core client for 30 sec - exiting
12:37:25 (8433): No heartbeat from core client for 30 sec - exiting
12:37:26 (8433): No heartbeat from core client for 30 sec - exiting
12:37:27 (8433): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08631810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
08:33:42 (390): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0886a810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x083d6810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
23:40:21 (10767): No heartbeat from core client for 30 sec - exiting
22:53:27 (10767): No heartbeat from core client for 30 sec - exiting
09:03:31 (10767): No heartbeat from core client for 30 sec - exiting
09:04:24 (10767): No heartbeat from core client for 30 sec - exiting
09:04:25 (10767): No heartbeat from core client for 30 sec - exiting
09:04:26 (10767): No heartbeat from core client for 30 sec - exiting
01:51:56 (10767): No heartbeat from core client for 30 sec - exiting
01:52:07 (10767): No heartbeat from core client for 30 sec - exiting
01:52:29 (10767): No heartbeat from core client for 30 sec - exiting
01:52:30 (10767): No heartbeat from core client for 30 sec - exiting
01:52:31 (10767): No heartbeat from core client for 30 sec - exiting
01:52:32 (10767): No heartbeat from core client for 30 sec - exiting
01:52:33 (10767): No heartbeat from core client for 30 sec - exiting
01:52:34 (10767): No heartbeat from core client for 30 sec - exiting
01:52:35 (10767): No heartbeat from core client for 30 sec - exiting
01:52:36 (10767): No heartbeat from core client for 30 sec - exiting
11:53:54 (10767): No heartbeat from core client for 30 sec - exiting
11:54:02 (10767): No heartbeat from core client for 30 sec - exiting
02:23:29 (10767): No heartbeat from core client for 30 sec - exiting
02:25:32 (10767): No heartbeat from core client for 30 sec - exiting
02:25:33 (10767): No heartbeat from core client for 30 sec - exiting
02:25:34 (10767): No heartbeat from core client for 30 sec - exiting
02:25:35 (10767): No heartbeat from core client for 30 sec - exiting
02:25:36 (10767): No heartbeat from core client for 30 sec - exiting
02:25:37 (10767): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09dc4810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
11:25:19 (2681): No heartbeat from core client for 30 sec - exiting
01:35:12 (2681): No heartbeat from core client for 30 sec - exiting
18:21:43 (2681): No heartbeat from core client for 30 sec - exiting
08:29:44 (2681): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0853f810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
08:40:58 (30173): No heartbeat from core client for 30 sec - exiting
22:49:07 (30173): No heartbeat from core client for 30 sec - exiting
08:52:58 (30173): No heartbeat from core client for 30 sec - exiting
08:53:07 (30173): No heartbeat from core client for 30 sec - exiting
08:53:08 (30173): No heartbeat from core client for 30 sec - exiting
08:53:09 (30173): No heartbeat from core client for 30 sec - exiting
08:53:10 (30173): No heartbeat from core client for 30 sec - exiting
08:53:11 (30173): No heartbeat from core client for 30 sec - exiting
08:54:04 (30173): No heartbeat from core client for 30 sec - exiting
08:54:05 (30173): No heartbeat from core client for 30 sec - exiting
08:54:06 (30173): No heartbeat from core client for 30 sec - exiting
08:54:07 (30173): No heartbeat from core client for 30 sec - exiting
08:54:08 (30173): No heartbeat from core client for 30 sec - exiting
08:54:09 (30173): No heartbeat from core client for 30 sec - exiting
08:54:11 (30173): No heartbeat from core client for 30 sec - exiting
08:54:12 (30173): No heartbeat from core client for 30 sec - exiting
08:54:13 (30173): No heartbeat from core client for 30 sec - exiting
08:54:14 (30173): No heartbeat from core client for 30 sec - exiting
08:54:15 (30173): No heartbeat from core client for 30 sec - exiting
08:54:16 (30173): No heartbeat from core client for 30 sec - exiting
08:54:17 (30173): No heartbeat from core client for 30 sec - exiting
08:54:18 (30173): No heartbeat from core client for 30 sec - exiting
08:54:19 (30173): No heartbeat from core client for 30 sec - exiting
08:54:20 (30173): No heartbeat from core client for 30 sec - exiting
08:54:21 (30173): No heartbeat from core client for 30 sec - exiting
08:54:22 (30173): No heartbeat from core client for 30 sec - exiting
08:54:23 (30173): No heartbeat from core client for 30 sec - exiting
08:54:24 (30173): No heartbeat from core client for 30 sec - exiting
08:54:25 (30173): No heartbeat from core client for 30 sec - exiting
08:54:26 (30173): No heartbeat from core client for 30 sec - exiting
08:54:27 (30173): No heartbeat from core client for 30 sec - exiting
08:54:28 (30173): No heartbeat from core client for 30 sec - exiting
08:54:29 (30173): No heartbeat from core client for 30 sec - exiting
08:54:30 (30173): No heartbeat from core client for 30 sec - exiting
08:54:31 (30173): No heartbeat from core client for 30 sec - exiting
08:54:32 (30173): No heartbeat from core client for 30 sec - exiting
08:54:33 (30173): No heartbeat from core client for 30 sec - exiting
08:54:34 (30173): No heartbeat from core client for 30 sec - exiting
08:54:35 (30173): No heartbeat from core client for 30 sec - exiting
23:33:14 (30173): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0979f810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
04:17:35 (25268): No heartbeat from core client for 30 sec - exiting
08:32:17 (25268): No heartbeat from core client for 30 sec - exiting
08:33:21 (25268): No heartbeat from core client for 30 sec - exiting
08:33:22 (25268): No heartbeat from core client for 30 sec - exiting
08:33:23 (25268): No heartbeat from core client for 30 sec - exiting
08:33:24 (25268): No heartbeat from core client for 30 sec - exiting
08:33:25 (25268): No heartbeat from core client for 30 sec - exiting
08:33:26 (25268): No heartbeat from core client for 30 sec - exiting
08:33:27 (25268): No heartbeat from core client for 30 sec - exiting
08:33:28 (25268): No heartbeat from core client for 30 sec - exiting
08:33:29 (25268): No heartbeat from core client for 30 sec - exiting
08:33:31 (25268): No heartbeat from core client for 30 sec - exiting
08:33:32 (25268): No heartbeat from core client for 30 sec - exiting
08:33:33 (25268): No heartbeat from core client for 30 sec - exiting
08:33:34 (25268): No heartbeat from core client for 30 sec - exiting
08:33:35 (25268): No heartbeat from core client for 30 sec - exiting
08:33:36 (25268): No heartbeat from core client for 30 sec - exiting
08:33:37 (25268): No heartbeat from core client for 30 sec - exiting
08:33:38 (25268): No heartbeat from core client for 30 sec - exiting
08:33:39 (25268): No heartbeat from core client for 30 sec - exiting
08:33:40 (25268): No heartbeat from core client for 30 sec - exiting
08:33:41 (25268): No heartbeat from core client for 30 sec - exiting
08:33:42 (25268): No heartbeat from core client for 30 sec - exiting
08:34:17 (25268): No heartbeat from core client for 30 sec - exiting
08:34:18 (25268): No heartbeat from core client for 30 sec - exiting
08:34:19 (25268): No heartbeat from core client for 30 sec - exiting
08:34:20 (25268): No heartbeat from core client for 30 sec - exiting
08:34:21 (25268): No heartbeat from core client for 30 sec - exiting
08:34:22 (25268): No heartbeat from core client for 30 sec - exiting
08:34:26 (25268): No heartbeat from core client for 30 sec - exiting
08:34:27 (25268): No heartbeat from core client for 30 sec - exiting
08:34:28 (25268): No heartbeat from core client for 30 sec - exiting
08:34:33 (25268): No heartbeat from core client for 30 sec - exiting
08:34:34 (25268): No heartbeat from core client for 30 sec - exiting
08:34:35 (25268): No heartbeat from core client for 30 sec - exiting
08:34:36 (25268): No heartbeat from core client for 30 sec - exiting
08:34:37 (25268): No heartbeat from core client for 30 sec - exiting
08:34:38 (25268): No heartbeat from core client for 30 sec - exiting
08:34:39 (25268): No heartbeat from core client for 30 sec - exiting
08:34:40 (25268): No heartbeat from core client for 30 sec - exiting
08:34:41 (25268): No heartbeat from core client for 30 sec - exiting
08:34:42 (25268): No heartbeat from core client for 30 sec - exiting
08:34:43 (25268): No heartbeat from core client for 30 sec - exiting
08:34:44 (25268): No heartbeat from core client for 30 sec - exiting
08:34:45 (25268): No heartbeat from core client for 30 sec - exiting
08:34:48 (25268): No heartbeat from core client for 30 sec - exiting
08:34:49 (25268): No heartbeat from core client for 30 sec - exiting
08:34:50 (25268): No heartbeat from core client for 30 sec - exiting
08:34:51 (25268): No heartbeat from core client for 30 sec - exiting
08:34:52 (25268): No heartbeat from core client for 30 sec - exiting
08:34:53 (25268): No heartbeat from core client for 30 sec - exiting
08:34:54 (25268): No heartbeat from core client for 30 sec - exiting
08:34:58 (25268): No heartbeat from core client for 30 sec - exiting
08:34:59 (25268): No heartbeat from core client for 30 sec - exiting
08:35:00 (25268): No heartbeat from core client for 30 sec - exiting
08:35:01 (25268): No heartbeat from core client for 30 sec - exiting
08:35:02 (25268): No heartbeat from core client for 30 sec - exiting
08:35:03 (25268): No heartbeat from core client for 30 sec - exiting
08:35:04 (25268): No heartbeat from core client for 30 sec - exiting
08:35:10 (25268): No heartbeat from core client for 30 sec - exiting
08:35:11 (25268): No heartbeat from core client for 30 sec - exiting
08:35:12 (25268): No heartbeat from core client for 30 sec - exiting
08:35:13 (25268): No heartbeat from core client for 30 sec - exiting
08:35:14 (25268): No heartbeat from core client for 30 sec - exiting
08:35:15 (25268): No heartbeat from core client for 30 sec - exiting
08:35:16 (25268): No heartbeat from core client for 30 sec - exiting
08:35:17 (25268): No heartbeat from core client for 30 sec - exiting
08:35:18 (25268): No heartbeat from core client for 30 sec - exiting
08:35:19 (25268): No heartbeat from core client for 30 sec - exiting
08:35:20 (25268): No heartbeat from core client for 30 sec - exiting
08:35:52 (25268): No heartbeat from core client for 30 sec - exiting
08:35:53 (25268): No heartbeat from core client for 30 sec - exiting
08:35:54 (25268): No heartbeat from core client for 30 sec - exiting
08:35:55 (25268): No heartbeat from core client for 30 sec - exiting
08:35:56 (25268): No heartbeat from core client for 30 sec - exiting
08:35:57 (25268): No heartbeat from core client for 30 sec - exiting
08:35:58 (25268): No heartbeat from core client for 30 sec - exiting
08:35:59 (25268): No heartbeat from core client for 30 sec - exiting
08:36:00 (25268): No heartbeat from core client for 30 sec - exiting
08:36:01 (25268): No heartbeat from core client for 30 sec - exiting
08:36:02 (25268): No heartbeat from core client for 30 sec - exiting
08:36:03 (25268): No heartbeat from core client for 30 sec - exiting
08:36:04 (25268): No heartbeat from core client for 30 sec - exiting
08:36:05 (25268): No heartbeat from core client for 30 sec - exiting
08:36:06 (25268): No heartbeat from core client for 30 sec - exiting
08:36:07 (25268): No heartbeat from core client for 30 sec - exiting
08:36:08 (25268): No heartbeat from core client for 30 sec - exiting
08:36:09 (25268): No heartbeat from core client for 30 sec - exiting
08:36:10 (25268): No heartbeat from core client for 30 sec - exiting
19:02:36 (25268): No heartbeat from core client for 30 sec - exiting
19:02:44 (25268): No heartbeat from core client for 30 sec - exiting
19:02:45 (25268): No heartbeat from core client for 30 sec - exiting
19:02:46 (25268): No heartbeat from core client for 30 sec - exiting
19:02:47 (25268): No heartbeat from core client for 30 sec - exiting
19:02:48 (25268): No heartbeat from core client for 30 sec - exiting
19:02:49 (25268): No heartbeat from core client for 30 sec - exiting
19:02:55 (25268): No heartbeat from core client for 30 sec - exiting
19:02:56 (25268): No heartbeat from core client for 30 sec - exiting
19:02:57 (25268): No heartbeat from core client for 30 sec - exiting
19:02:58 (25268): No heartbeat from core client for 30 sec - exiting
19:02:59 (25268): No heartbeat from core client for 30 sec - exiting
19:03:00 (25268): No heartbeat from core client for 30 sec - exiting
19:03:01 (25268): No heartbeat from core client for 30 sec - exiting
19:03:02 (25268): No heartbeat from core client for 30 sec - exiting
19:03:03 (25268): No heartbeat from core client for 30 sec - exiting
19:03:04 (25268): No heartbeat from core client for 30 sec - exiting
19:03:05 (25268): No heartbeat from core client for 30 sec - exiting
19:03:06 (25268): No heartbeat from core client for 30 sec - exiting
19:03:07 (25268): No heartbeat from core client for 30 sec - exiting
19:03:08 (25268): No heartbeat from core client for 30 sec - exiting
19:03:09 (25268): No heartbeat from core client for 30 sec - exiting
19:03:10 (25268): No heartbeat from core client for 30 sec - exiting
19:03:11 (25268): No heartbeat from core client for 30 sec - exiting
19:03:12 (25268): No heartbeat from core client for 30 sec - exiting
19:03:13 (25268): No heartbeat from core client for 30 sec - exiting
19:03:14 (25268): No heartbeat from core client for 30 sec - exiting
19:03:15 (25268): No heartbeat from core client for 30 sec - exiting
08:00:50 (25268): No heartbeat from core client for 30 sec - exiting
08:01:09 (25268): No heartbeat from core client for 30 sec - exiting
08:01:42 (25268): No heartbeat from core client for 30 sec - exiting
08:01:43 (25268): No heartbeat from core client for 30 sec - exiting
08:01:44 (25268): No heartbeat from core client for 30 sec - exiting
08:01:45 (25268): No heartbeat from core client for 30 sec - exiting
08:01:46 (25268): No heartbeat from core client for 30 sec - exiting
08:01:47 (25268): No heartbeat from core client for 30 sec - exiting
08:01:48 (25268): No heartbeat from core client for 30 sec - exiting
08:01:49 (25268): No heartbeat from core client for 30 sec - exiting
08:01:50 (25268): No heartbeat from core client for 30 sec - exiting
08:01:51 (25268): No heartbeat from core client for 30 sec - exiting
08:01:52 (25268): No heartbeat from core client for 30 sec - exiting
08:01:53 (25268): No heartbeat from core client for 30 sec - exiting
08:01:54 (25268): No heartbeat from core client for 30 sec - exiting
08:01:55 (25268): No heartbeat from core client for 30 sec - exiting
08:01:56 (25268): No heartbeat from core client for 30 sec - exiting
08:01:57 (25268): No heartbeat from core client for 30 sec - exiting
08:01:58 (25268): No heartbeat from core client for 30 sec - exiting
08:01:59 (25268): No heartbeat from core client for 30 sec - exiting
08:02:00 (25268): No heartbeat from core client for 30 sec - exiting
08:02:01 (25268): No heartbeat from core client for 30 sec - exiting
08:02:02 (25268): No heartbeat from core client for 30 sec - exiting
08:02:03 (25268): No heartbeat from core client for 30 sec - exiting
08:02:04 (25268): No heartbeat from core client for 30 sec - exiting
08:02:19 (25268): No heartbeat from core client for 30 sec - exiting
08:02:22 (25268): No heartbeat from core client for 30 sec - exiting
08:02:23 (25268): No heartbeat from core client for 30 sec - exiting
08:02:24 (25268): No heartbeat from core client for 30 sec - exiting
08:02:25 (25268): No heartbeat from core client for 30 sec - exiting
08:02:26 (25268): No heartbeat from core client for 30 sec - exiting
08:02:27 (25268): No heartbeat from core client for 30 sec - exiting
08:02:28 (25268): No heartbeat from core client for 30 sec - exiting
08:02:29 (25268): No heartbeat from core client for 30 sec - exiting
08:02:30 (25268): No heartbeat from core client for 30 sec - exiting
08:02:31 (25268): No heartbeat from core client for 30 sec - exiting
08:02:32 (25268): No heartbeat from core client for 30 sec - exiting
08:02:33 (25268): No heartbeat from core client for 30 sec - exiting
08:02:34 (25268): No heartbeat from core client for 30 sec - exiting
08:02:35 (25268): No heartbeat from core client for 30 sec - exiting
08:02:36 (25268): No heartbeat from core client for 30 sec - exiting
08:02:37 (25268): No heartbeat from core client for 30 sec - exiting
08:02:38 (25268): No heartbeat from core client for 30 sec - exiting
08:02:39 (25268): No heartbeat from core client for 30 sec - exiting
08:02:40 (25268): No heartbeat from core client for 30 sec - exiting
08:02:41 (25268): No heartbeat from core client for 30 sec - exiting
08:02:42 (25268): No heartbeat from core client for 30 sec - exiting
08:02:43 (25268): No heartbeat from core client for 30 sec - exiting
08:02:44 (25268): No heartbeat from core client for 30 sec - exiting
08:02:45 (25268): No heartbeat from core client for 30 sec - exiting
08:02:46 (25268): No heartbeat from core client for 30 sec - exiting
08:02:47 (25268): No heartbeat from core client for 30 sec - exiting
08:02:48 (25268): No heartbeat from core client for 30 sec - exiting
08:02:51 (25268): No heartbeat from core client for 30 sec - exiting
08:02:52 (25268): No heartbeat from core client for 30 sec - exiting
08:02:53 (25268): No heartbeat from core client for 30 sec - exiting
08:02:54 (25268): No heartbeat from core client for 30 sec - exiting
08:02:55 (25268): No heartbeat from core client for 30 sec - exiting
08:02:56 (25268): No heartbeat from core client for 30 sec - exiting
08:02:57 (25268): No heartbeat from core client for 30 sec - exiting
08:02:58 (25268): No heartbeat from core client for 30 sec - exiting
08:02:59 (25268): No heartbeat from core client for 30 sec - exiting
08:03:00 (25268): No heartbeat from core client for 30 sec - exiting
08:03:01 (25268): No heartbeat from core client for 30 sec - exiting
22:08:34 (25268): No heartbeat from core client for 30 sec - exiting
22:08:39 (25268): No heartbeat from core client for 30 sec - exiting
22:08:40 (25268): No heartbeat from core client for 30 sec - exiting
22:08:41 (25268): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08f05810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
00:04:54 (6316): No heartbeat from core client for 30 sec - exiting
09:51:51 (6316): No heartbeat from core client for 30 sec - exiting
09:52:22 (6316): No heartbeat from core client for 30 sec - exiting
09:52:23 (6316): No heartbeat from core client for 30 sec - exiting
09:52:24 (6316): No heartbeat from core client for 30 sec - exiting
09:52:25 (6316): No heartbeat from core client for 30 sec - exiting
09:52:26 (6316): No heartbeat from core client for 30 sec - exiting
09:52:37 (6316): No heartbeat from core client for 30 sec - exiting
09:52:38 (6316): No heartbeat from core client for 30 sec - exiting
09:52:39 (6316): No heartbeat from core client for 30 sec - exiting
09:52:40 (6316): No heartbeat from core client for 30 sec - exiting
09:52:41 (6316): No heartbeat from core client for 30 sec - exiting
09:52:42 (6316): No heartbeat from core client for 30 sec - exiting
09:52:43 (6316): No heartbeat from core client for 30 sec - exiting
09:52:44 (6316): No heartbeat from core client for 30 sec - exiting
09:52:45 (6316): No heartbeat from core client for 30 sec - exiting
09:52:46 (6316): No heartbeat from core client for 30 sec - exiting
09:52:49 (6316): No heartbeat from core client for 30 sec - exiting
09:52:50 (6316): No heartbeat from core client for 30 sec - exiting
09:52:51 (6316): No heartbeat from core client for 30 sec - exiting
09:52:52 (6316): No heartbeat from core client for 30 sec - exiting
09:52:53 (6316): No heartbeat from core client for 30 sec - exiting
09:52:57 (6316): No heartbeat from core client for 30 sec - exiting
09:52:58 (6316): No heartbeat from core client for 30 sec - exiting
09:52:59 (6316): No heartbeat from core client for 30 sec - exiting
09:53:00 (6316): No heartbeat from core client for 30 sec - exiting
09:53:01 (6316): No heartbeat from core client for 30 sec - exiting
09:53:02 (6316): No heartbeat from core client for 30 sec - exiting
09:53:03 (6316): No heartbeat from core client for 30 sec - exiting
09:53:04 (6316): No heartbeat from core client for 30 sec - exiting
09:53:05 (6316): No heartbeat from core client for 30 sec - exiting
09:53:06 (6316): No heartbeat from core client for 30 sec - exiting
09:53:07 (6316): No heartbeat from core client for 30 sec - exiting
09:53:08 (6316): No heartbeat from core client for 30 sec - exiting
09:53:09 (6316): No heartbeat from core client for 30 sec - exiting
09:53:10 (6316): No heartbeat from core client for 30 sec - exiting
09:53:11 (6316): No heartbeat from core client for 30 sec - exiting
09:53:12 (6316): No heartbeat from core client for 30 sec - exiting
09:53:13 (6316): No heartbeat from core client for 30 sec - exiting
09:53:14 (6316): No heartbeat from core client for 30 sec - exiting
09:53:15 (6316): No heartbeat from core client for 30 sec - exiting
09:53:16 (6316): No heartbeat from core client for 30 sec - exiting
09:53:17 (6316): No heartbeat from core client for 30 sec - exiting
09:53:18 (6316): No heartbeat from core client for 30 sec - exiting
09:53:20 (6316): No heartbeat from core client for 30 sec - exiting
09:53:21 (6316): No heartbeat from core client for 30 sec - exiting
09:53:22 (6316): No heartbeat from core client for 30 sec - exiting
09:53:23 (6316): No heartbeat from core client for 30 sec - exiting
09:53:24 (6316): No heartbeat from core client for 30 sec - exiting
09:53:25 (6316): No heartbeat from core client for 30 sec - exiting
09:53:26 (6316): No heartbeat from core client for 30 sec - exiting
09:53:27 (6316): No heartbeat from core client for 30 sec - exiting
09:53:28 (6316): No heartbeat from core client for 30 sec - exiting
09:53:29 (6316): No heartbeat from core client for 30 sec - exiting
09:54:00 (6316): No heartbeat from core client for 30 sec - exiting
09:54:01 (6316): No heartbeat from core client for 30 sec - exiting
09:54:02 (6316): No heartbeat from core client for 30 sec - exiting
09:54:03 (6316): No heartbeat from core client for 30 sec - exiting
09:54:04 (6316): No heartbeat from core client for 30 sec - exiting
09:54:05 (6316): No heartbeat from core client for 30 sec - exiting
09:54:06 (6316): No heartbeat from core client for 30 sec - exiting
09:54:07 (6316): No heartbeat from core client for 30 sec - exiting
09:54:08 (6316): No heartbeat from core client for 30 sec - exiting
09:54:09 (6316): No heartbeat from core client for 30 sec - exiting
09:54:10 (6316): No heartbeat from core client for 30 sec - exiting
09:54:11 (6316): No heartbeat from core client for 30 sec - exiting
09:54:12 (6316): No heartbeat from core client for 30 sec - exiting
09:54:13 (6316): No heartbeat from core client for 30 sec - exiting
09:54:14 (6316): No heartbeat from core client for 30 sec - exiting
09:54:15 (6316): No heartbeat from core client for 30 sec - exiting
09:54:16 (6316): No heartbeat from core client for 30 sec - exiting
09:54:17 (6316): No heartbeat from core client for 30 sec - exiting
09:54:18 (6316): No heartbeat from core client for 30 sec - exiting
09:54:19 (6316): No heartbeat from core client for 30 sec - exiting
09:54:50 (6316): No heartbeat from core client for 30 sec - exiting
09:54:51 (6316): No heartbeat from core client for 30 sec - exiting
09:54:52 (6316): No heartbeat from core client for 30 sec - exiting
09:54:53 (6316): No heartbeat from core client for 30 sec - exiting
09:54:54 (6316): No heartbeat from core client for 30 sec - exiting
09:54:55 (6316): No heartbeat from core client for 30 sec - exiting
09:54:56 (6316): No heartbeat from core client for 30 sec - exiting
09:54:57 (6316): No heartbeat from core client for 30 sec - exiting
09:54:58 (6316): No heartbeat from core client for 30 sec - exiting
09:54:59 (6316): No heartbeat from core client for 30 sec - exiting
09:55:00 (6316): No heartbeat from core client for 30 sec - exiting
09:55:01 (6316): No heartbeat from core client for 30 sec - exiting
09:55:02 (6316): No heartbeat from core client for 30 sec - exiting
09:55:03 (6316): No heartbeat from core client for 30 sec - exiting
09:55:04 (6316): No heartbeat from core client for 30 sec - exiting
09:55:05 (6316): No heartbeat from core client for 30 sec - exiting
07:31:20 (6316): No heartbeat from core client for 30 sec - exiting
07:31:25 (6316): No heartbeat from core client for 30 sec - exiting
07:31:26 (6316): No heartbeat from core client for 30 sec - exiting
07:31:27 (6316): No heartbeat from core client for 30 sec - exiting
07:31:28 (6316): No heartbeat from core client for 30 sec - exiting
07:31:29 (6316): No heartbeat from core client for 30 sec - exiting
07:31:30 (6316): No heartbeat from core client for 30 sec - exiting
07:31:31 (6316): No heartbeat from core client for 30 sec - exiting
09:13:05 (6316): No heartbeat from core client for 30 sec - exiting
22:06:53 (6316): No heartbeat from core client for 30 sec - exiting
22:07:41 (6316): No heartbeat from core client for 30 sec - exiting
22:07:42 (6316): No heartbeat from core client for 30 sec - exiting
22:07:43 (6316): No heartbeat from core client for 30 sec - exiting
22:07:44 (6316): No heartbeat from core client for 30 sec - exiting
22:07:45 (6316): No heartbeat from core client for 30 sec - exiting
22:07:46 (6316): No heartbeat from core client for 30 sec - exiting
22:07:47 (6316): No heartbeat from core client for 30 sec - exiting
22:07:48 (6316): No heartbeat from core client for 30 sec - exiting
22:07:49 (6316): No heartbeat from core client for 30 sec - exiting
22:07:50 (6316): No heartbeat from core client for 30 sec - exiting
22:07:51 (6316): No heartbeat from core client for 30 sec - exiting
22:07:52 (6316): No heartbeat from core client for 30 sec - exiting
22:07:53 (6316): No heartbeat from core client for 30 sec - exiting
22:07:54 (6316): No heartbeat from core client for 30 sec - exiting
22:07:55 (6316): No heartbeat from core client for 30 sec - exiting
22:07:56 (6316): No heartbeat from core client for 30 sec - exiting
22:07:57 (6316): No heartbeat from core client for 30 sec - exiting
22:07:58 (6316): No heartbeat from core client for 30 sec - exiting
22:07:59 (6316): No heartbeat from core client for 30 sec - exiting
22:08:00 (6316): No heartbeat from core client for 30 sec - exiting
22:08:01 (6316): No heartbeat from core client for 30 sec - exiting
22:08:02 (6316): No heartbeat from core client for 30 sec - exiting
22:08:03 (6316): No heartbeat from core client for 30 sec - exiting
22:08:04 (6316): No heartbeat from core client for 30 sec - exiting
22:08:05 (6316): No heartbeat from core client for 30 sec - exiting
22:08:06 (6316): No heartbeat from core client for 30 sec - exiting
22:08:07 (6316): No heartbeat from core client for 30 sec - exiting
22:08:12 (6316): No heartbeat from core client for 30 sec - exiting
22:08:13 (6316): No heartbeat from core client for 30 sec - exiting
22:08:14 (6316): No heartbeat from core client for 30 sec - exiting
22:08:15 (6316): No heartbeat from core client for 30 sec - exiting
22:08:16 (6316): No heartbeat from core client for 30 sec - exiting
22:08:17 (6316): No heartbeat from core client for 30 sec - exiting
22:08:18 (6316): No heartbeat from core client for 30 sec - exiting
22:08:19 (6316): No heartbeat from core client for 30 sec - exiting
22:08:20 (6316): No heartbeat from core client for 30 sec - exiting
22:08:21 (6316): No heartbeat from core client for 30 sec - exiting
22:08:22 (6316): No heartbeat from core client for 30 sec - exiting
22:08:23 (6316): No heartbeat from core client for 30 sec - exiting
22:08:24 (6316): No heartbeat from core client for 30 sec - exiting
09:41:13 (6316): No heartbeat from core client for 30 sec - exiting
22:08:39 (6316): No heartbeat from core client for 30 sec - exiting
22:10:43 (6316): No heartbeat from core client for 30 sec - exiting
22:10:44 (6316): No heartbeat from core client for 30 sec - exiting
22:10:45 (6316): No heartbeat from core client for 30 sec - exiting
22:10:46 (6316): No heartbeat from core client for 30 sec - exiting
22:10:47 (6316): No heartbeat from core client for 30 sec - exiting
22:10:48 (6316): No heartbeat from core client for 30 sec - exiting
22:10:49 (6316): No heartbeat from core client for 30 sec - exiting
22:10:50 (6316): No heartbeat from core client for 30 sec - exiting
22:10:51 (6316): No heartbeat from core client for 30 sec - exiting
22:10:52 (6316): No heartbeat from core client for 30 sec - exiting
22:10:53 (6316): No heartbeat from core client for 30 sec - exiting
22:11:03 (6316): No heartbeat from core client for 30 sec - exiting
22:11:04 (6316): No heartbeat from core client for 30 sec - exiting
22:11:05 (6316): No heartbeat from core client for 30 sec - exiting
22:11:06 (6316): No heartbeat from core client for 30 sec - exiting
22:11:08 (6316): No heartbeat from core client for 30 sec - exiting
22:11:09 (6316): No heartbeat from core client for 30 sec - exiting
22:11:10 (6316): No heartbeat from core client for 30 sec - exiting
22:11:11 (6316): No heartbeat from core client for 30 sec - exiting
22:11:15 (6316): No heartbeat from core client for 30 sec - exiting
22:11:16 (6316): No heartbeat from core client for 30 sec - exiting
22:11:17 (6316): No heartbeat from core client for 30 sec - exiting
22:11:18 (6316): No heartbeat from core client for 30 sec - exiting
22:11:19 (6316): No heartbeat from core client for 30 sec - exiting
22:11:20 (6316): No heartbeat from core client for 30 sec - exiting
22:11:21 (6316): No heartbeat from core client for 30 sec - exiting
22:11:22 (6316): No heartbeat from core client for 30 sec - exiting
22:11:23 (6316): No heartbeat from core client for 30 sec - exiting
22:11:24 (6316): No heartbeat from core client for 30 sec - exiting
22:11:25 (6316): No heartbeat from core client for 30 sec - exiting
22:11:26 (6316): No heartbeat from core client for 30 sec - exiting
22:11:27 (6316): No heartbeat from core client for 30 sec - exiting
22:11:28 (6316): No heartbeat from core client for 30 sec - exiting
22:11:29 (6316): No heartbeat from core client for 30 sec - exiting
22:11:30 (6316): No heartbeat from core client for 30 sec - exiting
22:11:31 (6316): No heartbeat from core client for 30 sec - exiting
22:11:32 (6316): No heartbeat from core client for 30 sec - exiting
22:11:33 (6316): No heartbeat from core client for 30 sec - exiting
22:11:34 (6316): No heartbeat from core client for 30 sec - exiting
22:11:35 (6316): No heartbeat from core client for 30 sec - exiting
22:11:36 (6316): No heartbeat from core client for 30 sec - exiting
22:11:37 (6316): No heartbeat from core client for 30 sec - exiting
22:11:38 (6316): No heartbeat from core client for 30 sec - exiting
22:11:39 (6316): No heartbeat from core client for 30 sec - exiting
22:11:40 (6316): No heartbeat from core client for 30 sec - exiting
22:11:42 (6316): No heartbeat from core client for 30 sec - exiting
22:11:43 (6316): No heartbeat from core client for 30 sec - exiting
22:11:44 (6316): No heartbeat from core client for 30 sec - exiting
22:11:45 (6316): No heartbeat from core client for 30 sec - exiting
22:11:46 (6316): No heartbeat from core client for 30 sec - exiting
22:11:47 (6316): No heartbeat from core client for 30 sec - exiting
22:11:48 (6316): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0852d810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
05:41:26 (29333): No heartbeat from core client for 30 sec - exiting
22:09:40 (29333): No heartbeat from core client for 30 sec - exiting
22:10:35 (29333): No heartbeat from core client for 30 sec - exiting
22:10:36 (29333): No heartbeat from core client for 30 sec - exiting
22:10:37 (29333): No heartbeat from core client for 30 sec - exiting
22:10:38 (29333): No heartbeat from core client for 30 sec - exiting
22:10:39 (29333): No heartbeat from core client for 30 sec - exiting
22:10:40 (29333): No heartbeat from core client for 30 sec - exiting
22:10:41 (29333): No heartbeat from core client for 30 sec - exiting
22:10:42 (29333): No heartbeat from core client for 30 sec - exiting
22:10:43 (29333): No heartbeat from core client for 30 sec - exiting
22:10:44 (29333): No heartbeat from core client for 30 sec - exiting
22:10:45 (29333): No heartbeat from core client for 30 sec - exiting
22:10:46 (29333): No heartbeat from core client for 30 sec - exiting
22:10:47 (29333): No heartbeat from core client for 30 sec - exiting
22:10:48 (29333): No heartbeat from core client for 30 sec - exiting
22:10:49 (29333): No heartbeat from core client for 30 sec - exiting
22:10:50 (29333): No heartbeat from core client for 30 sec - exiting
22:10:51 (29333): No heartbeat from core client for 30 sec - exiting
22:10:52 (29333): No heartbeat from core client for 30 sec - exiting
22:10:53 (29333): No heartbeat from core client for 30 sec - exiting
22:10:54 (29333): No heartbeat from core client for 30 sec - exiting
22:10:55 (29333): No heartbeat from core client for 30 sec - exiting
22:10:56 (29333): No heartbeat from core client for 30 sec - exiting
22:10:57 (29333): No heartbeat from core client for 30 sec - exiting
22:10:58 (29333): No heartbeat from core client for 30 sec - exiting
22:10:59 (29333): No heartbeat from core client for 30 sec - exiting
22:11:00 (29333): No heartbeat from core client for 30 sec - exiting
22:11:01 (29333): No heartbeat from core client for 30 sec - exiting
22:12:10 (29333): No heartbeat from core client for 30 sec - exiting
22:12:11 (29333): No heartbeat from core client for 30 sec - exiting
22:12:12 (29333): No heartbeat from core client for 30 sec - exiting
22:12:13 (29333): No heartbeat from core client for 30 sec - exiting
22:12:14 (29333): No heartbeat from core client for 30 sec - exiting
22:12:15 (29333): No heartbeat from core client for 30 sec - exiting
22:12:16 (29333): No heartbeat from core client for 30 sec - exiting
22:12:17 (29333): No heartbeat from core client for 30 sec - exiting
22:12:18 (29333): No heartbeat from core client for 30 sec - exiting
22:12:19 (29333): No heartbeat from core client for 30 sec - exiting
22:12:20 (29333): No heartbeat from core client for 30 sec - exiting
22:12:21 (29333): No heartbeat from core client for 30 sec - exiting
22:12:22 (29333): No heartbeat from core client for 30 sec - exiting
22:12:25 (29333): No heartbeat from core client for 30 sec - exiting
22:12:26 (29333): No heartbeat from core client for 30 sec - exiting
22:12:27 (29333): No heartbeat from core client for 30 sec - exiting
22:12:28 (29333): No heartbeat from core client for 30 sec - exiting
22:12:29 (29333): No heartbeat from core client for 30 sec - exiting
22:12:30 (29333): No heartbeat from core client for 30 sec - exiting
22:12:31 (29333): No heartbeat from core client for 30 sec - exiting
22:12:32 (29333): No heartbeat from core client for 30 sec - exiting
22:12:33 (29333): No heartbeat from core client for 30 sec - exiting
22:12:34 (29333): No heartbeat from core client for 30 sec - exiting
22:12:35 (29333): No heartbeat from core client for 30 sec - exiting
22:12:36 (29333): No heartbeat from core client for 30 sec - exiting
22:12:37 (29333): No heartbeat from core client for 30 sec - exiting
22:12:38 (29333): No heartbeat from core client for 30 sec - exiting
22:12:39 (29333): No heartbeat from core client for 30 sec - exiting
22:12:40 (29333): No heartbeat from core client for 30 sec - exiting
22:12:41 (29333): No heartbeat from core client for 30 sec - exiting
22:12:42 (29333): No heartbeat from core client for 30 sec - exiting
22:12:43 (29333): No heartbeat from core client for 30 sec - exiting
22:12:44 (29333): No heartbeat from core client for 30 sec - exiting
22:12:45 (29333): No heartbeat from core client for 30 sec - exiting
22:12:46 (29333): No heartbeat from core client for 30 sec - exiting
22:12:47 (29333): No heartbeat from core client for 30 sec - exiting
22:12:48 (29333): No heartbeat from core client for 30 sec - exiting
08:05:46 (29333): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08d32810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
22:10:46 (10435): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0985b810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08a7c810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
18:24:08 (2066): No heartbeat from core client for 30 sec - exiting
08:38:04 (2066): No heartbeat from core client for 30 sec - exiting
08:38:17 (2066): No heartbeat from core client for 30 sec - exiting
08:38:18 (2066): No heartbeat from core client for 30 sec - exiting
20:46:36 (2066): No heartbeat from core client for 30 sec - exiting
06:27:20 (2066): No heartbeat from core client for 30 sec - exiting
06:27:53 (2066): No heartbeat from core client for 30 sec - exiting
06:27:54 (2066): No heartbeat from core client for 30 sec - exiting
06:27:55 (2066): No heartbeat from core client for 30 sec - exiting
06:27:56 (2066): No heartbeat from core client for 30 sec - exiting
06:27:57 (2066): No heartbeat from core client for 30 sec - exiting
06:27:58 (2066): No heartbeat from core client for 30 sec - exiting
06:27:59 (2066): No heartbeat from core client for 30 sec - exiting
06:28:00 (2066): No heartbeat from core client for 30 sec - exiting
06:28:31 (2066): No heartbeat from core client for 30 sec - exiting
06:28:32 (2066): No heartbeat from core client for 30 sec - exiting
06:28:33 (2066): No heartbeat from core client for 30 sec - exiting
06:28:34 (2066): No heartbeat from core client for 30 sec - exiting
06:51:39 (2066): No heartbeat from core client for 30 sec - exiting
06:52:27 (2066): No heartbeat from core client for 30 sec - exiting
06:52:28 (2066): No heartbeat from core client for 30 sec - exiting
06:52:59 (2066): No heartbeat from core client for 30 sec - exiting
06:53:00 (2066): No heartbeat from core client for 30 sec - exiting
06:53:01 (2066): No heartbeat from core client for 30 sec - exiting
06:53:02 (2066): No heartbeat from core client for 30 sec - exiting
06:53:03 (2066): No heartbeat from core client for 30 sec - exiting
06:53:04 (2066): No heartbeat from core client for 30 sec - exiting
06:53:05 (2066): No heartbeat from core client for 30 sec - exiting
06:53:06 (2066): No heartbeat from core client for 30 sec - exiting
06:53:07 (2066): No heartbeat from core client for 30 sec - exiting
06:53:08 (2066): No heartbeat from core client for 30 sec - exiting
06:53:09 (2066): No heartbeat from core client for 30 sec - exiting
06:53:10 (2066): No heartbeat from core client for 30 sec - exiting
06:53:41 (2066): No heartbeat from core client for 30 sec - exiting
06:53:42 (2066): No heartbeat from core client for 30 sec - exiting
06:53:43 (2066): No heartbeat from core client for 30 sec - exiting
06:53:44 (2066): No heartbeat from core client for 30 sec - exiting
06:53:45 (2066): No heartbeat from core client for 30 sec - exiting
06:53:46 (2066): No heartbeat from core client for 30 sec - exiting
06:53:47 (2066): No heartbeat from core client for 30 sec - exiting
06:53:48 (2066): No heartbeat from core client for 30 sec - exiting
06:53:49 (2066): No heartbeat from core client for 30 sec - exiting
06:53:50 (2066): No heartbeat from core client for 30 sec - exiting
06:53:51 (2066): No heartbeat from core client for 30 sec - exiting
06:53:52 (2066): No heartbeat from core client for 30 sec - exiting
06:53:53 (2066): No heartbeat from core client for 30 sec - exiting
06:53:54 (2066): No heartbeat from core client for 30 sec - exiting
06:53:55 (2066): No heartbeat from core client for 30 sec - exiting
06:53:56 (2066): No heartbeat from core client for 30 sec - exiting
06:53:57 (2066): No heartbeat from core client for 30 sec - exiting
06:53:58 (2066): No heartbeat from core client for 30 sec - exiting
06:53:59 (2066): No heartbeat from core client for 30 sec - exiting
03:05:59 (2066): No heartbeat from core client for 30 sec - exiting
03:06:13 (2066): No heartbeat from core client for 30 sec - exiting
03:06:14 (2066): No heartbeat from core client for 30 sec - exiting
03:06:15 (2066): No heartbeat from core client for 30 sec - exiting
03:06:16 (2066): No heartbeat from core client for 30 sec - exiting
03:06:17 (2066): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0893d810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08937810 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
Signal 3 received, exiting...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
22 Jan 2013 22:11:06 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 285,120 728,837 2.5562
19 Jan 2013 23:12:40 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 259,200 626,510 2.4171
05 Jan 2013 11:11:19 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 233,280 591,580 2.5359
02 Jan 2013 10:45:36 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 207,360 524,966 2.5317
27 Dec 2012 21:56:40 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 181,440 461,017 2.5409
25 Dec 2012 07:37:15 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 155,520 396,830 2.5516
20 Dec 2012 12:06:17 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 129,600 333,296 2.5717
14 Dec 2012 03:20:35 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 103,680 269,617 2.6005
07 Dec 2012 13:47:50 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 77,760 204,632 2.6316
03 Dec 2012 06:48:59 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 51,840 138,957 2.6805
25 Nov 2012 03:04:14 1242159 15451191 hadcm3n_zmh3_1880_40_008250218_1 25,920 67,651 2.6100


©2024 climateprediction.net