climateprediction.net home page
Task 16050729

Task 16050729

Name hadcm3n_48b7_1980_40_008340403_4
Workunit 8491264
Created 30 Sep 2013, 11:58:22 UTC
Sent 30 Sep 2013, 12:15:00 UTC
Report deadline 30 Dec 2013, 19:42:11 UTC
Received 13 Nov 2013, 21:22:35 UTC
Server state Over
Outcome Computation error
Client state Aborted by user
Exit status 203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID 1295603
Run time 4 days 10 hours 46 min 37 sec
CPU time 4 days 8 hours 23 min 24 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 2.75 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
aborted by user
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/48b7ko.pji2c10 is not a valid UM file.
Error converting file to netcdf: dataout/48b7ko.pji2c10
Error: Input file: dataout/48b7ko.pii2c10 is not a valid UM file.
Error converting file to netcdf: dataout/48b7ko.pii2c10
Error: Input file: dataout/48b7ko.pfi2c10 is not a valid UM file.
Error converting file to netcdf: dataout/48b7ko.pfi2c10
Error: Input file: dataout/48b7ka.phi2c10 is not a valid UM file.
Error converting file to netcdf: dataout/48b7ka.phi2c10
Error: Input file: dataout/48b7ka.pgi2c10 is not a valid UM file.
Error converting file to netcdf: dataout/48b7ka.pgi2c10
Error: Input file: dataout/48b7ka.pei2c10 is not a valid UM file.
Error converting file to netcdf: dataout/48b7ka.pei2c10
Error: Input file: dataout/48b7ka.pdi2c10 is not a valid UM file.
Error converting file to netcdf: dataout/48b7ka.pdi2c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
17:09:03 (4897): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
01:10:26 (8656): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:51:53 (8898): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:51:57 (8898): No heartbeat from core client for 30 sec - exiting
05:51:58 (8898): No heartbeat from core client for 30 sec - exiting
05:51:59 (8898): No heartbeat from core client for 30 sec - exiting
05:52:00 (8898): No heartbeat from core client for 30 sec - exiting
05:52:01 (8898): No heartbeat from core client for 30 sec - exiting
05:52:02 (8898): No heartbeat from core client for 30 sec - exiting
06:04:38 (9218): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:13:09 (9238): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:19:28 (9244): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:24:57 (9253): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:36:08 (9281): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:38:27 (9307): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:42:09 (9311): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:48:08 (9315): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:48:54 (9460): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:51:41 (9464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:55:07 (9468): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:58:17 (9472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:58:18 (9472): No heartbeat from core client for 30 sec - exiting
06:58:19 (9472): No heartbeat from core client for 30 sec - exiting
06:58:20 (9472): No heartbeat from core client for 30 sec - exiting
06:59:00 (9476): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:01:00 (9480): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:03:40 (9484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:04:20 (9489): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:07:22 (9493): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:09:33 (9497): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:11:23 (9502): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:15:27 (9506): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:17:42 (9511): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:18:45 (9518): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:20:38 (9523): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:20:40 (9523): No heartbeat from core client for 30 sec - exiting
07:23:18 (9527): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:26:35 (9532): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:29:50 (9536): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:32:44 (9541): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:35:29 (9545): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:38:11 (9550): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:38:54 (9554): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:40:34 (9559): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:43:41 (9563): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:46:44 (9568): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:49:23 (9572): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:52:04 (9577): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09d348b0 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
14:03:46 (10323): No heartbeat from core client for 30 sec - exiting
14:03:48 (10323): No heartbeat from core client for 30 sec - exiting
14:03:49 (10323): No heartbeat from core client for 30 sec - exiting
14:03:50 (10323): No heartbeat from core client for 30 sec - exiting
14:03:51 (10323): No heartbeat from core client for 30 sec - exiting
14:03:52 (10323): No heartbeat from core client for 30 sec - exiting
14:03:53 (10323): No heartbeat from core client for 30 sec - exiting
14:03:54 (10323): No heartbeat from core client for 30 sec - exiting
14:03:55 (10323): No heartbeat from core client for 30 sec - exiting
14:03:56 (10323): No heartbeat from core client for 30 sec - exiting
14:08:15 (10323): No heartbeat from core client for 30 sec - exiting
14:08:16 (10323): No heartbeat from core client for 30 sec - exiting
14:08:17 (10323): No heartbeat from core client for 30 sec - exiting
14:08:18 (10323): No heartbeat from core client for 30 sec - exiting
14:08:19 (10323): No heartbeat from core client for 30 sec - exiting
14:08:20 (10323): No heartbeat from core client for 30 sec - exiting
14:08:21 (10323): No heartbeat from core client for 30 sec - exiting
14:08:22 (10323): No heartbeat from core client for 30 sec - exiting
14:09:15 (10323): No heartbeat from core client for 30 sec - exiting
14:09:16 (10323): No heartbeat from core client for 30 sec - exiting
14:09:17 (10323): No heartbeat from core client for 30 sec - exiting
14:09:18 (10323): No heartbeat from core client for 30 sec - exiting
14:09:19 (10323): No heartbeat from core client for 30 sec - exiting
14:09:20 (10323): No heartbeat from core client for 30 sec - exiting
14:09:21 (10323): No heartbeat from core client for 30 sec - exiting
14:09:22 (10323): No heartbeat from core client for 30 sec - exiting
14:09:23 (10323): No heartbeat from core client for 30 sec - exiting
14:09:24 (10323): No heartbeat from core client for 30 sec - exiting
14:13:26 (10323): No heartbeat from core client for 30 sec - exiting
14:13:27 (10323): No heartbeat from core client for 30 sec - exiting
14:13:28 (10323): No heartbeat from core client for 30 sec - exiting
14:13:29 (10323): No heartbeat from core client for 30 sec - exiting
14:13:30 (10323): No heartbeat from core client for 30 sec - exiting
14:13:31 (10323): No heartbeat from core client for 30 sec - exiting
14:13:32 (10323): No heartbeat from core client for 30 sec - exiting
14:13:33 (10323): No heartbeat from core client for 30 sec - exiting
14:13:34 (10323): No heartbeat from core client for 30 sec - exiting
14:13:35 (10323): No heartbeat from core client for 30 sec - exiting
14:17:07 (10323): No heartbeat from core client for 30 sec - exiting
14:17:08 (10323): No heartbeat from core client for 30 sec - exiting
14:17:09 (10323): No heartbeat from core client for 30 sec - exiting
14:17:10 (10323): No heartbeat from core client for 30 sec - exiting
14:17:11 (10323): No heartbeat from core client for 30 sec - exiting
14:17:12 (10323): No heartbeat from core client for 30 sec - exiting
14:17:13 (10323): No heartbeat from core client for 30 sec - exiting
14:17:14 (10323): No heartbeat from core client for 30 sec - exiting
14:17:15 (10323): No heartbeat from core client for 30 sec - exiting
14:17:16 (10323): No heartbeat from core client for 30 sec - exiting
14:17:17 (10323): No heartbeat from core client for 30 sec - exiting
14:17:18 (10323): No heartbeat from core client for 30 sec - exiting
14:17:19 (10323): No heartbeat from core client for 30 sec - exiting
14:17:20 (10323): No heartbeat from core client for 30 sec - exiting
14:17:21 (10323): No heartbeat from core client for 30 sec - exiting
14:17:22 (10323): No heartbeat from core client for 30 sec - exiting
14:17:23 (10323): No heartbeat from core client for 30 sec - exiting
14:17:24 (10323): No heartbeat from core client for 30 sec - exiting
14:17:25 (10323): No heartbeat from core client for 30 sec - exiting
14:17:26 (10323): No heartbeat from core client for 30 sec - exiting
14:17:27 (10323): No heartbeat from core client for 30 sec - exiting
14:17:28 (10323): No heartbeat from core client for 30 sec - exiting
14:17:29 (10323): No heartbeat from core client for 30 sec - exiting
14:17:30 (10323): No heartbeat from core client for 30 sec - exiting
14:17:31 (10323): No heartbeat from core client for 30 sec - exiting
14:17:32 (10323): No heartbeat from core client for 30 sec - exiting
14:17:33 (10323): No heartbeat from core client for 30 sec - exiting
14:17:34 (10323): No heartbeat from core client for 30 sec - exiting
14:17:35 (10323): No heartbeat from core client for 30 sec - exiting
14:17:36 (10323): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08a053a0 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
06:10:23 (10634): No heartbeat from core client for 30 sec - exiting
06:10:24 (10634): No heartbeat from core client for 30 sec - exiting
06:10:25 (10634): No heartbeat from core client for 30 sec - exiting
22:12:45 (10634): No heartbeat from core client for 30 sec - exiting
22:12:47 (10634): No heartbeat from core client for 30 sec - exiting
22:12:48 (10634): No heartbeat from core client for 30 sec - exiting
22:12:49 (10634): No heartbeat from core client for 30 sec - exiting
22:12:50 (10634): No heartbeat from core client for 30 sec - exiting
22:12:51 (10634): No heartbeat from core client for 30 sec - exiting
22:12:52 (10634): No heartbeat from core client for 30 sec - exiting
22:12:53 (10634): No heartbeat from core client for 30 sec - exiting
22:13:26 (10634): No heartbeat from core client for 30 sec - exiting
22:13:28 (10634): No heartbeat from core client for 30 sec - exiting
22:13:29 (10634): No heartbeat from core client for 30 sec - exiting
22:13:30 (10634): No heartbeat from core client for 30 sec - exiting
22:13:31 (10634): No heartbeat from core client for 30 sec - exiting
22:13:32 (10634): No heartbeat from core client for 30 sec - exiting
22:13:33 (10634): No heartbeat from core client for 30 sec - exiting
22:13:34 (10634): No heartbeat from core client for 30 sec - exiting
22:13:35 (10634): No heartbeat from core client for 30 sec - exiting
22:22:57 (10634): No heartbeat from core client for 30 sec - exiting
22:22:58 (10634): No heartbeat from core client for 30 sec - exiting
22:22:59 (10634): No heartbeat from core client for 30 sec - exiting
22:23:00 (10634): No heartbeat from core client for 30 sec - exiting
22:23:01 (10634): No heartbeat from core client for 30 sec - exiting
22:23:02 (10634): No heartbeat from core client for 30 sec - exiting
22:23:03 (10634): No heartbeat from core client for 30 sec - exiting
22:23:04 (10634): No heartbeat from core client for 30 sec - exiting
22:23:05 (10634): No heartbeat from core client for 30 sec - exiting
22:23:06 (10634): No heartbeat from core client for 30 sec - exiting
22:23:07 (10634): No heartbeat from core client for 30 sec - exiting
22:24:08 (10634): No heartbeat from core client for 30 sec - exiting
22:24:09 (10634): No heartbeat from core client for 30 sec - exiting
22:24:10 (10634): No heartbeat from core client for 30 sec - exiting
22:24:11 (10634): No heartbeat from core client for 30 sec - exiting
22:24:12 (10634): No heartbeat from core client for 30 sec - exiting
22:24:13 (10634): No heartbeat from core client for 30 sec - exiting
22:24:14 (10634): No heartbeat from core client for 30 sec - exiting
22:24:15 (10634): No heartbeat from core client for 30 sec - exiting
22:24:16 (10634): No heartbeat from core client for 30 sec - exiting
22:24:17 (10634): No heartbeat from core client for 30 sec - exiting
22:26:06 (10634): No heartbeat from core client for 30 sec - exiting
22:26:07 (10634): No heartbeat from core client for 30 sec - exiting
22:26:08 (10634): No heartbeat from core client for 30 sec - exiting
22:26:09 (10634): No heartbeat from core client for 30 sec - exiting
22:26:10 (10634): No heartbeat from core client for 30 sec - exiting
22:26:11 (10634): No heartbeat from core client for 30 sec - exiting
22:26:12 (10634): No heartbeat from core client for 30 sec - exiting
22:26:13 (10634): No heartbeat from core client for 30 sec - exiting
22:26:14 (10634): No heartbeat from core client for 30 sec - exiting
22:26:15 (10634): No heartbeat from core client for 30 sec - exiting
22:26:16 (10634): No heartbeat from core client for 30 sec - exiting
22:27:13 (10634): No heartbeat from core client for 30 sec - exiting
22:27:14 (10634): No heartbeat from core client for 30 sec - exiting
22:27:15 (10634): No heartbeat from core client for 30 sec - exiting
22:27:16 (10634): No heartbeat from core client for 30 sec - exiting
22:27:17 (10634): No heartbeat from core client for 30 sec - exiting
22:27:18 (10634): No heartbeat from core client for 30 sec - exiting
22:27:19 (10634): No heartbeat from core client for 30 sec - exiting
22:27:20 (10634): No heartbeat from core client for 30 sec - exiting
22:27:21 (10634): No heartbeat from core client for 30 sec - exiting
22:27:22 (10634): No heartbeat from core client for 30 sec - exiting
22:27:54 (10634): No heartbeat from core client for 30 sec - exiting
22:27:55 (10634): No heartbeat from core client for 30 sec - exiting
22:27:56 (10634): No heartbeat from core client for 30 sec - exiting
22:27:57 (10634): No heartbeat from core client for 30 sec - exiting
22:27:58 (10634): No heartbeat from core client for 30 sec - exiting
22:27:59 (10634): No heartbeat from core client for 30 sec - exiting
22:28:00 (10634): No heartbeat from core client for 30 sec - exiting
22:28:01 (10634): No heartbeat from core client for 30 sec - exiting
22:28:02 (10634): No heartbeat from core client for 30 sec - exiting
22:28:03 (10634): No heartbeat from core client for 30 sec - exiting
22:29:25 (10634): No heartbeat from core client for 30 sec - exiting
22:29:26 (10634): No heartbeat from core client for 30 sec - exiting
22:29:28 (10634): No heartbeat from core client for 30 sec - exiting
22:29:29 (10634): No heartbeat from core client for 30 sec - exiting
22:29:30 (10634): No heartbeat from core client for 30 sec - exiting
23:07:36 (10634): No heartbeat from core client for 30 sec - exiting
23:07:37 (10634): No heartbeat from core client for 30 sec - exiting
23:07:38 (10634): No heartbeat from core client for 30 sec - exiting
23:07:39 (10634): No heartbeat from core client for 30 sec - exiting
23:07:40 (10634): No heartbeat from core client for 30 sec - exiting
23:07:41 (10634): No heartbeat from core client for 30 sec - exiting
23:07:42 (10634): No heartbeat from core client for 30 sec - exiting
23:07:43 (10634): No heartbeat from core client for 30 sec - exiting
23:07:44 (10634): No heartbeat from core client for 30 sec - exiting
23:07:45 (10634): No heartbeat from core client for 30 sec - exiting
23:08:16 (10634): No heartbeat from core client for 30 sec - exiting
23:08:17 (10634): No heartbeat from core client for 30 sec - exiting
23:08:18 (10634): No heartbeat from core client for 30 sec - exiting
23:08:19 (10634): No heartbeat from core client for 30 sec - exiting
23:08:20 (10634): No heartbeat from core client for 30 sec - exiting
23:08:21 (10634): No heartbeat from core client for 30 sec - exiting
23:08:22 (10634): No heartbeat from core client for 30 sec - exiting
23:08:23 (10634): No heartbeat from core client for 30 sec - exiting
23:08:24 (10634): No heartbeat from core client for 30 sec - exiting
23:08:25 (10634): No heartbeat from core client for 30 sec - exiting
23:10:57 (10634): No heartbeat from core client for 30 sec - exiting
23:10:58 (10634): No heartbeat from core client for 30 sec - exiting
23:10:59 (10634): No heartbeat from core client for 30 sec - exiting
23:11:00 (10634): No heartbeat from core client for 30 sec - exiting
23:11:01 (10634): No heartbeat from core client for 30 sec - exiting
23:11:02 (10634): No heartbeat from core client for 30 sec - exiting
23:11:03 (10634): No heartbeat from core client for 30 sec - exiting
23:11:04 (10634): No heartbeat from core client for 30 sec - exiting
23:11:05 (10634): No heartbeat from core client for 30 sec - exiting
23:11:06 (10634): No heartbeat from core client for 30 sec - exiting
23:11:07 (10634): No heartbeat from core client for 30 sec - exiting
23:14:44 (10634): No heartbeat from core client for 30 sec - exiting
23:14:45 (10634): No heartbeat from core client for 30 sec - exiting
23:14:46 (10634): No heartbeat from core client for 30 sec - exiting
23:14:47 (10634): No heartbeat from core client for 30 sec - exiting
23:14:48 (10634): No heartbeat from core client for 30 sec - exiting
23:14:49 (10634): No heartbeat from core client for 30 sec - exiting
23:14:50 (10634): No heartbeat from core client for 30 sec - exiting
23:14:51 (10634): No heartbeat from core client for 30 sec - exiting
23:14:52 (10634): No heartbeat from core client for 30 sec - exiting
23:14:53 (10634): No heartbeat from core client for 30 sec - exiting
23:50:36 (10634): No heartbeat from core client for 30 sec - exiting
23:50:37 (10634): No heartbeat from core client for 30 sec - exiting
23:50:38 (10634): No heartbeat from core client for 30 sec - exiting
23:50:39 (10634): No heartbeat from core client for 30 sec - exiting
23:50:40 (10634): No heartbeat from core client for 30 sec - exiting
23:50:41 (10634): No heartbeat from core client for 30 sec - exiting
23:50:42 (10634): No heartbeat from core client for 30 sec - exiting
23:50:43 (10634): No heartbeat from core client for 30 sec - exiting
23:50:44 (10634): No heartbeat from core client for 30 sec - exiting
23:50:45 (10634): No heartbeat from core client for 30 sec - exiting
23:50:46 (10634): No heartbeat from core client for 30 sec - exiting
23:51:48 (10634): No heartbeat from core client for 30 sec - exiting
23:51:49 (10634): No heartbeat from core client for 30 sec - exiting
23:51:50 (10634): No heartbeat from core client for 30 sec - exiting
23:51:51 (10634): No heartbeat from core client for 30 sec - exiting
23:51:52 (10634): No heartbeat from core client for 30 sec - exiting
23:51:53 (10634): No heartbeat from core client for 30 sec - exiting
23:51:54 (10634): No heartbeat from core client for 30 sec - exiting
23:51:55 (10634): No heartbeat from core client for 30 sec - exiting
23:51:56 (10634): No heartbeat from core client for 30 sec - exiting
23:51:57 (10634): No heartbeat from core client for 30 sec - exiting
23:51:58 (10634): No heartbeat from core client for 30 sec - exiting
23:53:38 (10634): No heartbeat from core client for 30 sec - exiting
23:53:39 (10634): No heartbeat from core client for 30 sec - exiting
23:53:40 (10634): No heartbeat from core client for 30 sec - exiting
23:53:41 (10634): No heartbeat from core client for 30 sec - exiting
23:53:42 (10634): No heartbeat from core client for 30 sec - exiting
23:53:43 (10634): No heartbeat from core client for 30 sec - exiting
23:53:44 (10634): No heartbeat from core client for 30 sec - exiting
23:53:45 (10634): No heartbeat from core client for 30 sec - exiting
23:53:46 (10634): No heartbeat from core client for 30 sec - exiting
23:53:47 (10634): No heartbeat from core client for 30 sec - exiting
23:53:48 (10634): No heartbeat from core client for 30 sec - exiting
05:32:20 (10634): No heartbeat from core client for 30 sec - exiting
05:32:23 (10634): No heartbeat from core client for 30 sec - exiting
05:32:24 (10634): No heartbeat from core client for 30 sec - exiting
05:32:25 (10634): No heartbeat from core client for 30 sec - exiting
05:32:26 (10634): No heartbeat from core client for 30 sec - exiting
05:32:27 (10634): No heartbeat from core client for 30 sec - exiting
05:32:28 (10634): No heartbeat from core client for 30 sec - exiting
05:32:29 (10634): No heartbeat from core client for 30 sec - exiting
05:32:30 (10634): No heartbeat from core client for 30 sec - exiting
05:32:31 (10634): No heartbeat from core client for 30 sec - exiting
05:32:32 (10634): No heartbeat from core client for 30 sec - exiting
05:32:33 (10634): No heartbeat from core client for 30 sec - exiting
05:32:34 (10634): No heartbeat from core client for 30 sec - exiting
05:32:35 (10634): No heartbeat from core client for 30 sec - exiting
05:32:36 (10634): No heartbeat from core client for 30 sec - exiting
05:32:37 (10634): No heartbeat from core client for 30 sec - exiting
05:32:38 (10634): No heartbeat from core client for 30 sec - exiting
05:32:39 (10634): No heartbeat from core client for 30 sec - exiting
05:32:40 (10634): No heartbeat from core client for 30 sec - exiting
05:32:41 (10634): No heartbeat from core client for 30 sec - exiting
05:32:42 (10634): No heartbeat from core client for 30 sec - exiting
05:32:43 (10634): No heartbeat from core client for 30 sec - exiting
05:32:44 (10634): No heartbeat from core client for 30 sec - exiting
05:32:45 (10634): No heartbeat from core client for 30 sec - exiting
05:32:46 (10634): No heartbeat from core client for 30 sec - exiting
05:32:47 (10634): No heartbeat from core client for 30 sec - exiting
05:32:48 (10634): No heartbeat from core client for 30 sec - exiting
05:32:49 (10634): No heartbeat from core client for 30 sec - exiting
05:32:50 (10634): No heartbeat from core client for 30 sec - exiting
05:32:51 (10634): No heartbeat from core client for 30 sec - exiting
05:32:52 (10634): No heartbeat from core client for 30 sec - exiting
05:32:53 (10634): No heartbeat from core client for 30 sec - exiting
05:32:54 (10634): No heartbeat from core client for 30 sec - exiting
05:32:55 (10634): No heartbeat from core client for 30 sec - exiting
05:32:56 (10634): No heartbeat from core client for 30 sec - exiting
05:32:57 (10634): No heartbeat from core client for 30 sec - exiting
05:32:58 (10634): No heartbeat from core client for 30 sec - exiting
05:32:59 (10634): No heartbeat from core client for 30 sec - exiting
05:33:00 (10634): No heartbeat from core client for 30 sec - exiting
05:33:01 (10634): No heartbeat from core client for 30 sec - exiting
05:33:02 (10634): No heartbeat from core client for 30 sec - exiting
05:33:03 (10634): No heartbeat from core client for 30 sec - exiting
05:33:04 (10634): No heartbeat from core client for 30 sec - exiting
05:35:26 (10634): No heartbeat from core client for 30 sec - exiting
05:35:27 (10634): No heartbeat from core client for 30 sec - exiting
05:35:28 (10634): No heartbeat from core client for 30 sec - exiting
05:35:29 (10634): No heartbeat from core client for 30 sec - exiting
05:35:56 (10634): No heartbeat from core client for 30 sec - exiting
11:16:33 (10634): No heartbeat from core client for 30 sec - exiting
11:16:35 (10634): No heartbeat from core client for 30 sec - exiting
11:16:36 (10634): No heartbeat from core client for 30 sec - exiting
11:16:37 (10634): No heartbeat from core client for 30 sec - exiting
11:16:38 (10634): No heartbeat from core client for 30 sec - exiting
11:16:39 (10634): No heartbeat from core client for 30 sec - exiting
11:16:40 (10634): No heartbeat from core client for 30 sec - exiting
11:16:41 (10634): No heartbeat from core client for 30 sec - exiting
11:16:42 (10634): No heartbeat from core client for 30 sec - exiting
11:16:43 (10634): No heartbeat from core client for 30 sec - exiting
11:17:30 (10634): No heartbeat from core client for 30 sec - exiting
11:17:31 (10634): No heartbeat from core client for 30 sec - exiting
11:17:32 (10634): No heartbeat from core client for 30 sec - exiting
11:17:33 (10634): No heartbeat from core client for 30 sec - exiting
11:17:34 (10634): No heartbeat from core client for 30 sec - exiting
11:17:35 (10634): No heartbeat from core client for 30 sec - exiting
11:36:34 (10634): No heartbeat from core client for 30 sec - exiting
11:36:35 (10634): No heartbeat from core client for 30 sec - exiting
11:36:36 (10634): No heartbeat from core client for 30 sec - exiting
11:36:37 (10634): No heartbeat from core client for 30 sec - exiting
11:36:38 (10634): No heartbeat from core client for 30 sec - exiting
11:36:39 (10634): No heartbeat from core client for 30 sec - exiting
11:36:40 (10634): No heartbeat from core client for 30 sec - exiting
11:36:41 (10634): No heartbeat from core client for 30 sec - exiting
11:36:42 (10634): No heartbeat from core client for 30 sec - exiting
11:36:43 (10634): No heartbeat from core client for 30 sec - exiting
11:36:44 (10634): No heartbeat from core client for 30 sec - exiting
11:39:10 (10634): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0a0f5390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
07:21:58 (1081): No heartbeat from core client for 30 sec - exiting
07:22:02 (1081): No heartbeat from core client for 30 sec - exiting
07:22:03 (1081): No heartbeat from core client for 30 sec - exiting
07:22:04 (1081): No heartbeat from core client for 30 sec - exiting
07:22:05 (1081): No heartbeat from core client for 30 sec - exiting
07:22:06 (1081): No heartbeat from core client for 30 sec - exiting
08:39:24 (1081): No heartbeat from core client for 30 sec - exiting
08:39:26 (1081): No heartbeat from core client for 30 sec - exiting
08:39:29 (1081): No heartbeat from core client for 30 sec - exiting
08:39:32 (1081): No heartbeat from core client for 30 sec - exiting
08:39:34 (1081): No heartbeat from core client for 30 sec - exiting
15:16:32 (1081): No heartbeat from core client for 30 sec - exiting
15:16:34 (1081): No heartbeat from core client for 30 sec - exiting
15:16:35 (1081): No heartbeat from core client for 30 sec - exiting
15:16:36 (1081): No heartbeat from core client for 30 sec - exiting
15:16:37 (1081): No heartbeat from core client for 30 sec - exiting
15:16:38 (1081): No heartbeat from core client for 30 sec - exiting
15:16:39 (1081): No heartbeat from core client for 30 sec - exiting
15:16:40 (1081): No heartbeat from core client for 30 sec - exiting
15:16:41 (1081): No heartbeat from core client for 30 sec - exiting
15:16:42 (1081): No heartbeat from core client for 30 sec - exiting
15:16:44 (1081): No heartbeat from core client for 30 sec - exiting
15:16:46 (1081): No heartbeat from core client for 30 sec - exiting
15:16:47 (1081): No heartbeat from core client for 30 sec - exiting
15:16:48 (1081): No heartbeat from core client for 30 sec - exiting
21:57:10 (1081): No heartbeat from core client for 30 sec - exiting
21:57:19 (1081): No heartbeat from core client for 30 sec - exiting
21:57:58 (1081): No heartbeat from core client for 30 sec - exiting
21:58:00 (1081): No heartbeat from core client for 30 sec - exiting
21:58:01 (1081): No heartbeat from core client for 30 sec - exiting
21:58:02 (1081): No heartbeat from core client for 30 sec - exiting
21:58:03 (1081): No heartbeat from core client for 30 sec - exiting
21:58:04 (1081): No heartbeat from core client for 30 sec - exiting
21:58:05 (1081): No heartbeat from core client for 30 sec - exiting
21:58:08 (1081): No heartbeat from core client for 30 sec - exiting
21:58:09 (1081): No heartbeat from core client for 30 sec - exiting
21:58:10 (1081): No heartbeat from core client for 30 sec - exiting
21:58:11 (1081): No heartbeat from core client for 30 sec - exiting
21:58:12 (1081): No heartbeat from core client for 30 sec - exiting
21:58:13 (1081): No heartbeat from core client for 30 sec - exiting
21:58:14 (1081): No heartbeat from core client for 30 sec - exiting
21:58:15 (1081): No heartbeat from core client for 30 sec - exiting
21:58:16 (1081): No heartbeat from core client for 30 sec - exiting
21:58:17 (1081): No heartbeat from core client for 30 sec - exiting
21:58:18 (1081): No heartbeat from core client for 30 sec - exiting
21:58:19 (1081): No heartbeat from core client for 30 sec - exiting
21:58:20 (1081): No heartbeat from core client for 30 sec - exiting
21:58:21 (1081): No heartbeat from core client for 30 sec - exiting
21:58:27 (1081): No heartbeat from core client for 30 sec - exiting
21:58:35 (1081): No heartbeat from core client for 30 sec - exiting
21:58:36 (1081): No heartbeat from core client for 30 sec - exiting
21:58:37 (1081): No heartbeat from core client for 30 sec - exiting
21:58:38 (1081): No heartbeat from core client for 30 sec - exiting
21:58:39 (1081): No heartbeat from core client for 30 sec - exiting
21:58:40 (1081): No heartbeat from core client for 30 sec - exiting
21:58:41 (1081): No heartbeat from core client for 30 sec - exiting
21:58:42 (1081): No heartbeat from core client for 30 sec - exiting
21:58:43 (1081): No heartbeat from core client for 30 sec - exiting
21:58:44 (1081): No heartbeat from core client for 30 sec - exiting
21:58:45 (1081): No heartbeat from core client for 30 sec - exiting
22:02:17 (1081): No heartbeat from core client for 30 sec - exiting
22:02:19 (1081): No heartbeat from core client for 30 sec - exiting
22:02:20 (1081): No heartbeat from core client for 30 sec - exiting
22:02:21 (1081): No heartbeat from core client for 30 sec - exiting
22:02:22 (1081): No heartbeat from core client for 30 sec - exiting
22:02:23 (1081): No heartbeat from core client for 30 sec - exiting
22:02:24 (1081): No heartbeat from core client for 30 sec - exiting
22:02:25 (1081): No heartbeat from core client for 30 sec - exiting
22:02:26 (1081): No heartbeat from core client for 30 sec - exiting
22:02:27 (1081): No heartbeat from core client for 30 sec - exiting
22:02:28 (1081): No heartbeat from core client for 30 sec - exiting
22:02:29 (1081): No heartbeat from core client for 30 sec - exiting
22:02:30 (1081): No heartbeat from core client for 30 sec - exiting
22:02:31 (1081): No heartbeat from core client for 30 sec - exiting
22:02:32 (1081): No heartbeat from core client for 30 sec - exiting
22:02:33 (1081): No heartbeat from core client for 30 sec - exiting
22:02:34 (1081): No heartbeat from core client for 30 sec - exiting
22:02:35 (1081): No heartbeat from core client for 30 sec - exiting
22:02:36 (1081): No heartbeat from core client for 30 sec - exiting
22:02:37 (1081): No heartbeat from core client for 30 sec - exiting
22:02:38 (1081): No heartbeat from core client for 30 sec - exiting
22:02:39 (1081): No heartbeat from core client for 30 sec - exiting
22:02:40 (1081): No heartbeat from core client for 30 sec - exiting
22:02:41 (1081): No heartbeat from core client for 30 sec - exiting
22:02:42 (1081): No heartbeat from core client for 30 sec - exiting
22:02:43 (1081): No heartbeat from core client for 30 sec - exiting
22:02:44 (1081): No heartbeat from core client for 30 sec - exiting
22:02:45 (1081): No heartbeat from core client for 30 sec - exiting
22:02:46 (1081): No heartbeat from core client for 30 sec - exiting
22:02:47 (1081): No heartbeat from core client for 30 sec - exiting
22:02:48 (1081): No heartbeat from core client for 30 sec - exiting
22:02:49 (1081): No heartbeat from core client for 30 sec - exiting
22:02:50 (1081): No heartbeat from core client for 30 sec - exiting
22:02:51 (1081): No heartbeat from core client for 30 sec - exiting
22:02:52 (1081): No heartbeat from core client for 30 sec - exiting
22:02:53 (1081): No heartbeat from core client for 30 sec - exiting
22:02:54 (1081): No heartbeat from core client for 30 sec - exiting
22:02:55 (1081): No heartbeat from core client for 30 sec - exiting
22:02:56 (1081): No heartbeat from core client for 30 sec - exiting
22:13:56 (1081): No heartbeat from core client for 30 sec - exiting
22:13:57 (1081): No heartbeat from core client for 30 sec - exiting
22:13:58 (1081): No heartbeat from core client for 30 sec - exiting
22:13:59 (1081): No heartbeat from core client for 30 sec - exiting
22:14:00 (1081): No heartbeat from core client for 30 sec - exiting
22:14:01 (1081): No heartbeat from core client for 30 sec - exiting
22:14:02 (1081): No heartbeat from core client for 30 sec - exiting
22:14:03 (1081): No heartbeat from core client for 30 sec - exiting
22:14:04 (1081): No heartbeat from core client for 30 sec - exiting
22:14:05 (1081): No heartbeat from core client for 30 sec - exiting
22:14:06 (1081): No heartbeat from core client for 30 sec - exiting
22:14:07 (1081): No heartbeat from core client for 30 sec - exiting
22:14:08 (1081): No heartbeat from core client for 30 sec - exiting
22:14:09 (1081): No heartbeat from core client for 30 sec - exiting
22:14:10 (1081): No heartbeat from core client for 30 sec - exiting
22:14:11 (1081): No heartbeat from core client for 30 sec - exiting
22:14:12 (1081): No heartbeat from core client for 30 sec - exiting
22:14:43 (1081): No heartbeat from core client for 30 sec - exiting
22:14:44 (1081): No heartbeat from core client for 30 sec - exiting
22:14:45 (1081): No heartbeat from core client for 30 sec - exiting
22:14:46 (1081): No heartbeat from core client for 30 sec - exiting
22:14:47 (1081): No heartbeat from core client for 30 sec - exiting
22:14:48 (1081): No heartbeat from core client for 30 sec - exiting
22:14:49 (1081): No heartbeat from core client for 30 sec - exiting
22:14:50 (1081): No heartbeat from core client for 30 sec - exiting
22:14:51 (1081): No heartbeat from core client for 30 sec - exiting
22:14:52 (1081): No heartbeat from core client for 30 sec - exiting
22:14:53 (1081): No heartbeat from core client for 30 sec - exiting
22:14:54 (1081): No heartbeat from core client for 30 sec - exiting
22:14:55 (1081): No heartbeat from core client for 30 sec - exiting
22:14:56 (1081): No heartbeat from core client for 30 sec - exiting
22:14:57 (1081): No heartbeat from core client for 30 sec - exiting
22:14:58 (1081): No heartbeat from core client for 30 sec - exiting
22:14:59 (1081): No heartbeat from core client for 30 sec - exiting
22:15:00 (1081): No heartbeat from core client for 30 sec - exiting
22:15:01 (1081): No heartbeat from core client for 30 sec - exiting
22:15:02 (1081): No heartbeat from core client for 30 sec - exiting
22:15:03 (1081): No heartbeat from core client for 30 sec - exiting
22:15:04 (1081): No heartbeat from core client for 30 sec - exiting
22:15:05 (1081): No heartbeat from core client for 30 sec - exiting
22:15:06 (1081): No heartbeat from core client for 30 sec - exiting
22:15:07 (1081): No heartbeat from core client for 30 sec - exiting
22:15:08 (1081): No heartbeat from core client for 30 sec - exiting
22:15:09 (1081): No heartbeat from core client for 30 sec - exiting
22:15:11 (1081): No heartbeat from core client for 30 sec - exiting
22:15:12 (1081): No heartbeat from core client for 30 sec - exiting
22:15:13 (1081): No heartbeat from core client for 30 sec - exiting
22:19:03 (1081): No heartbeat from core client for 30 sec - exiting
22:20:38 (1081): No heartbeat from core client for 30 sec - exiting
22:20:39 (1081): No heartbeat from core client for 30 sec - exiting
22:20:40 (1081): No heartbeat from core client for 30 sec - exiting
22:20:41 (1081): No heartbeat from core client for 30 sec - exiting
22:20:42 (1081): No heartbeat from core client for 30 sec - exiting
22:20:43 (1081): No heartbeat from core client for 30 sec - exiting
22:20:44 (1081): No heartbeat from core client for 30 sec - exiting
22:20:45 (1081): No heartbeat from core client for 30 sec - exiting
22:20:46 (1081): No heartbeat from core client for 30 sec - exiting
22:20:47 (1081): No heartbeat from core client for 30 sec - exiting
22:20:48 (1081): No heartbeat from core client for 30 sec - exiting
22:20:49 (1081): No heartbeat from core client for 30 sec - exiting
22:20:55 (1081): No heartbeat from core client for 30 sec - exiting
22:20:56 (1081): No heartbeat from core client for 30 sec - exiting
22:20:57 (1081): No heartbeat from core client for 30 sec - exiting
22:20:58 (1081): No heartbeat from core client for 30 sec - exiting
22:20:59 (1081): No heartbeat from core client for 30 sec - exiting
22:21:01 (1081): No heartbeat from core client for 30 sec - exiting
22:21:02 (1081): No heartbeat from core client for 30 sec - exiting
22:21:03 (1081): No heartbeat from core client for 30 sec - exiting
22:21:04 (1081): No heartbeat from core client for 30 sec - exiting
22:21:05 (1081): No heartbeat from core client for 30 sec - exiting
22:21:06 (1081): No heartbeat from core client for 30 sec - exiting
22:21:14 (1081): No heartbeat from core client for 30 sec - exiting
22:21:16 (1081): No heartbeat from core client for 30 sec - exiting
22:23:31 (1081): No heartbeat from core client for 30 sec - exiting
22:23:34 (1081): No heartbeat from core client for 30 sec - exiting
22:23:35 (1081): No heartbeat from core client for 30 sec - exiting
22:23:36 (1081): No heartbeat from core client for 30 sec - exiting
22:23:37 (1081): No heartbeat from core client for 30 sec - exiting
22:23:38 (1081): No heartbeat from core client for 30 sec - exiting
22:23:39 (1081): No heartbeat from core client for 30 sec - exiting
22:23:40 (1081): No heartbeat from core client for 30 sec - exiting
22:23:41 (1081): No heartbeat from core client for 30 sec - exiting
22:23:42 (1081): No heartbeat from core client for 30 sec - exiting
22:23:43 (1081): No heartbeat from core client for 30 sec - exiting
22:23:44 (1081): No heartbeat from core client for 30 sec - exiting
22:23:45 (1081): No heartbeat from core client for 30 sec - exiting
22:23:46 (1081): No heartbeat from core client for 30 sec - exiting
22:23:47 (1081): No heartbeat from core client for 30 sec - exiting
22:23:48 (1081): No heartbeat from core client for 30 sec - exiting
22:23:49 (1081): No heartbeat from core client for 30 sec - exiting
22:23:50 (1081): No heartbeat from core client for 30 sec - exiting
22:28:30 (1081): No heartbeat from core client for 30 sec - exiting
22:28:34 (1081): No heartbeat from core client for 30 sec - exiting
22:28:42 (1081): No heartbeat from core client for 30 sec - exiting
22:28:45 (1081): No heartbeat from core client for 30 sec - exiting
22:28:46 (1081): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08f4c390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09968390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09d96390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
14:13:48 (1263): No heartbeat from core client for 30 sec - exiting
14:13:49 (1263): No heartbeat from core client for 30 sec - exiting
14:13:50 (1263): No heartbeat from core client for 30 sec - exiting
14:13:51 (1263): No heartbeat from core client for 30 sec - exiting
14:13:52 (1263): No heartbeat from core client for 30 sec - exiting
14:13:53 (1263): No heartbeat from core client for 30 sec - exiting
14:13:54 (1263): No heartbeat from core client for 30 sec - exiting
14:13:55 (1263): No heartbeat from core client for 30 sec - exiting
14:13:56 (1263): No heartbeat from core client for 30 sec - exiting
14:13:57 (1263): No heartbeat from core client for 30 sec - exiting
14:13:58 (1263): No heartbeat from core client for 30 sec - exiting
14:23:58 (1263): No heartbeat from core client for 30 sec - exiting
14:29:55 (1263): No heartbeat from core client for 30 sec - exiting
14:29:56 (1263): No heartbeat from core client for 30 sec - exiting
14:29:57 (1263): No heartbeat from core client for 30 sec - exiting
14:29:58 (1263): No heartbeat from core client for 30 sec - exiting
14:29:59 (1263): No heartbeat from core client for 30 sec - exiting
14:30:00 (1263): No heartbeat from core client for 30 sec - exiting
14:30:01 (1263): No heartbeat from core client for 30 sec - exiting
14:30:02 (1263): No heartbeat from core client for 30 sec - exiting
14:30:03 (1263): No heartbeat from core client for 30 sec - exiting
14:30:04 (1263): No heartbeat from core client for 30 sec - exiting
14:30:05 (1263): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09ef1390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
11:47:55 (1899): No heartbeat from core client for 30 sec - exiting
12:21:10 (1899): No heartbeat from core client for 30 sec - exiting
12:46:30 (1899): No heartbeat from core client for 30 sec - exiting
13:13:20 (1899): No heartbeat from core client for 30 sec - exiting
13:15:35 (1899): No heartbeat from core client for 30 sec - exiting
13:19:01 (1899): No heartbeat from core client for 30 sec - exiting
13:19:02 (1899): No heartbeat from core client for 30 sec - exiting
13:19:03 (1899): No heartbeat from core client for 30 sec - exiting
13:19:04 (1899): No heartbeat from core client for 30 sec - exiting
13:19:05 (1899): No heartbeat from core client for 30 sec - exiting
13:19:06 (1899): No heartbeat from core client for 30 sec - exiting
13:19:07 (1899): No heartbeat from core client for 30 sec - exiting
13:19:08 (1899): No heartbeat from core client for 30 sec - exiting
13:19:09 (1899): No heartbeat from core client for 30 sec - exiting
13:19:10 (1899): No heartbeat from core client for 30 sec - exiting
13:19:11 (1899): No heartbeat from core client for 30 sec - exiting
13:19:42 (1899): No heartbeat from core client for 30 sec - exiting
13:19:43 (1899): No heartbeat from core client for 30 sec - exiting
13:19:44 (1899): No heartbeat from core client for 30 sec - exiting
13:19:45 (1899): No heartbeat from core client for 30 sec - exiting
13:19:46 (1899): No heartbeat from core client for 30 sec - exiting
13:19:47 (1899): No heartbeat from core client for 30 sec - exiting
13:19:48 (1899): No heartbeat from core client for 30 sec - exiting
13:19:49 (1899): No heartbeat from core client for 30 sec - exiting
13:19:50 (1899): No heartbeat from core client for 30 sec - exiting
13:19:51 (1899): No heartbeat from core client for 30 sec - exiting
13:19:52 (1899): No heartbeat from core client for 30 sec - exiting
13:22:09 (1899): No heartbeat from core client for 30 sec - exiting
13:22:10 (1899): No heartbeat from core client for 30 sec - exiting
13:22:11 (1899): No heartbeat from core client for 30 sec - exiting
13:22:12 (1899): No heartbeat from core client for 30 sec - exiting
13:22:13 (1899): No heartbeat from core client for 30 sec - exiting
13:22:14 (1899): No heartbeat from core client for 30 sec - exiting
13:22:15 (1899): No heartbeat from core client for 30 sec - exiting
13:22:16 (1899): No heartbeat from core client for 30 sec - exiting
13:22:17 (1899): No heartbeat from core client for 30 sec - exiting
13:22:18 (1899): No heartbeat from core client for 30 sec - exiting
13:24:49 (1899): No heartbeat from core client for 30 sec - exiting
13:28:53 (1899): No heartbeat from core client for 30 sec - exiting
13:28:54 (1899): No heartbeat from core client for 30 sec - exiting
13:28:55 (1899): No heartbeat from core client for 30 sec - exiting
13:28:56 (1899): No heartbeat from core client for 30 sec - exiting
13:28:57 (1899): No heartbeat from core client for 30 sec - exiting
13:28:58 (1899): No heartbeat from core client for 30 sec - exiting
13:28:59 (1899): No heartbeat from core client for 30 sec - exiting
13:29:00 (1899): No heartbeat from core client for 30 sec - exiting
13:29:01 (1899): No heartbeat from core client for 30 sec - exiting
13:29:02 (1899): No heartbeat from core client for 30 sec - exiting
13:29:03 (1899): No heartbeat from core client for 30 sec - exiting
13:29:04 (1899): No heartbeat from core client for 30 sec - exiting
13:31:15 (1899): No heartbeat from core client for 30 sec - exiting
13:31:16 (1899): No heartbeat from core client for 30 sec - exiting
13:31:17 (1899): No heartbeat from core client for 30 sec - exiting
13:31:18 (1899): No heartbeat from core client for 30 sec - exiting
13:31:19 (1899): No heartbeat from core client for 30 sec - exiting
13:31:20 (1899): No heartbeat from core client for 30 sec - exiting
13:31:21 (1899): No heartbeat from core client for 30 sec - exiting
13:31:22 (1899): No heartbeat from core client for 30 sec - exiting
13:31:23 (1899): No heartbeat from core client for 30 sec - exiting
13:31:24 (1899): No heartbeat from core client for 30 sec - exiting
13:33:31 (1899): No heartbeat from core client for 30 sec - exiting
13:34:54 (1899): No heartbeat from core client for 30 sec - exiting
13:34:55 (1899): No heartbeat from core client for 30 sec - exiting
13:34:56 (1899): No heartbeat from core client for 30 sec - exiting
13:34:57 (1899): No heartbeat from core client for 30 sec - exiting
13:34:58 (1899): No heartbeat from core client for 30 sec - exiting
13:34:59 (1899): No heartbeat from core client for 30 sec - exiting
13:35:00 (1899): No heartbeat from core client for 30 sec - exiting
13:35:01 (1899): No heartbeat from core client for 30 sec - exiting
13:35:02 (1899): No heartbeat from core client for 30 sec - exiting
13:35:03 (1899): No heartbeat from core client for 30 sec - exiting
13:35:34 (1899): No heartbeat from core client for 30 sec - exiting
13:37:52 (1899): No heartbeat from core client for 30 sec - exiting
13:37:53 (1899): No heartbeat from core client for 30 sec - exiting
13:37:54 (1899): No heartbeat from core client for 30 sec - exiting
13:37:55 (1899): No heartbeat from core client for 30 sec - exiting
13:37:56 (1899): No heartbeat from core client for 30 sec - exiting
13:37:57 (1899): No heartbeat from core client for 30 sec - exiting
13:37:58 (1899): No heartbeat from core client for 30 sec - exiting
13:37:59 (1899): No heartbeat from core client for 30 sec - exiting
13:38:00 (1899): No heartbeat from core client for 30 sec - exiting
13:38:01 (1899): No heartbeat from core client for 30 sec - exiting
13:38:02 (1899): No heartbeat from core client for 30 sec - exiting
13:40:18 (1899): No heartbeat from core client for 30 sec - exiting
13:40:19 (1899): No heartbeat from core client for 30 sec - exiting
13:40:20 (1899): No heartbeat from core client for 30 sec - exiting
13:40:21 (1899): No heartbeat from core client for 30 sec - exiting
13:40:22 (1899): No heartbeat from core client for 30 sec - exiting
13:40:23 (1899): No heartbeat from core client for 30 sec - exiting
13:40:24 (1899): No heartbeat from core client for 30 sec - exiting
13:40:25 (1899): No heartbeat from core client for 30 sec - exiting
13:40:26 (1899): No heartbeat from core client for 30 sec - exiting
13:40:27 (1899): No heartbeat from core client for 30 sec - exiting
13:40:28 (1899): No heartbeat from core client for 30 sec - exiting
13:42:27 (1899): No heartbeat from core client for 30 sec - exiting
13:42:28 (1899): No heartbeat from core client for 30 sec - exiting
13:42:29 (1899): No heartbeat from core client for 30 sec - exiting
13:42:30 (1899): No heartbeat from core client for 30 sec - exiting
13:42:31 (1899): No heartbeat from core client for 30 sec - exiting
13:42:32 (1899): No heartbeat from core client for 30 sec - exiting
13:42:33 (1899): No heartbeat from core client for 30 sec - exiting
13:42:34 (1899): No heartbeat from core client for 30 sec - exiting
13:42:35 (1899): No heartbeat from core client for 30 sec - exiting
13:42:36 (1899): No heartbeat from core client for 30 sec - exiting
13:42:37 (1899): No heartbeat from core client for 30 sec - exiting
13:43:13 (1899): No heartbeat from core client for 30 sec - exiting
13:43:14 (1899): No heartbeat from core client for 30 sec - exiting
13:43:15 (1899): No heartbeat from core client for 30 sec - exiting
13:43:16 (1899): No heartbeat from core client for 30 sec - exiting
13:43:17 (1899): No heartbeat from core client for 30 sec - exiting
13:43:18 (1899): No heartbeat from core client for 30 sec - exiting
13:43:19 (1899): No heartbeat from core client for 30 sec - exiting
13:43:20 (1899): No heartbeat from core client for 30 sec - exiting
13:43:21 (1899): No heartbeat from core client for 30 sec - exiting
13:43:22 (1899): No heartbeat from core client for 30 sec - exiting
13:43:23 (1899): No heartbeat from core client for 30 sec - exiting
13:45:15 (1899): No heartbeat from core client for 30 sec - exiting
13:45:16 (1899): No heartbeat from core client for 30 sec - exiting
13:45:17 (1899): No heartbeat from core client for 30 sec - exiting
13:45:18 (1899): No heartbeat from core client for 30 sec - exiting
13:45:19 (1899): No heartbeat from core client for 30 sec - exiting
13:45:20 (1899): No heartbeat from core client for 30 sec - exiting
13:45:21 (1899): No heartbeat from core client for 30 sec - exiting
13:45:22 (1899): No heartbeat from core client for 30 sec - exiting
13:45:23 (1899): No heartbeat from core client for 30 sec - exiting
13:45:24 (1899): No heartbeat from core client for 30 sec - exiting
13:47:30 (1899): No heartbeat from core client for 30 sec - exiting
13:49:09 (1899): No heartbeat from core client for 30 sec - exiting
13:50:48 (1899): No heartbeat from core client for 30 sec - exiting
13:52:26 (1899): No heartbeat from core client for 30 sec - exiting
13:54:12 (1899): No heartbeat from core client for 30 sec - exiting
13:54:13 (1899): No heartbeat from core client for 30 sec - exiting
13:54:14 (1899): No heartbeat from core client for 30 sec - exiting
13:54:15 (1899): No heartbeat from core client for 30 sec - exiting
13:54:16 (1899): No heartbeat from core client for 30 sec - exiting
13:54:17 (1899): No heartbeat from core client for 30 sec - exiting
13:54:18 (1899): No heartbeat from core client for 30 sec - exiting
13:54:19 (1899): No heartbeat from core client for 30 sec - exiting
13:54:20 (1899): No heartbeat from core client for 30 sec - exiting
13:54:21 (1899): No heartbeat from core client for 30 sec - exiting
13:54:22 (1899): No heartbeat from core client for 30 sec - exiting
04:04:00 (1899): No heartbeat from core client for 30 sec - exiting
04:04:01 (1899): No heartbeat from core client for 30 sec - exiting
04:04:02 (1899): No heartbeat from core client for 30 sec - exiting
04:04:03 (1899): No heartbeat from core client for 30 sec - exiting
04:04:04 (1899): No heartbeat from core client for 30 sec - exiting
04:04:05 (1899): No heartbeat from core client for 30 sec - exiting
04:04:06 (1899): No heartbeat from core client for 30 sec - exiting
04:04:07 (1899): No heartbeat from core client for 30 sec - exiting
04:04:08 (1899): No heartbeat from core client for 30 sec - exiting
04:04:09 (1899): No heartbeat from core client for 30 sec - exiting
04:04:10 (1899): No heartbeat from core client for 30 sec - exiting
06:54:26 (1899): No heartbeat from core client for 30 sec - exiting
06:54:27 (1899): No heartbeat from core client for 30 sec - exiting
06:54:28 (1899): No heartbeat from core client for 30 sec - exiting
06:54:29 (1899): No heartbeat from core client for 30 sec - exiting
06:54:30 (1899): No heartbeat from core client for 30 sec - exiting
06:54:31 (1899): No heartbeat from core client for 30 sec - exiting
06:54:32 (1899): No heartbeat from core client for 30 sec - exiting
06:54:33 (1899): No heartbeat from core client for 30 sec - exiting
06:54:34 (1899): No heartbeat from core client for 30 sec - exiting
06:54:35 (1899): No heartbeat from core client for 30 sec - exiting
06:54:36 (1899): No heartbeat from core client for 30 sec - exiting
07:19:30 (1899): No heartbeat from core client for 30 sec - exiting
16:23:00 (1899): No heartbeat from core client for 30 sec - exiting
16:26:41 (1899): No heartbeat from core client for 30 sec - exiting
16:26:42 (1899): No heartbeat from core client for 30 sec - exiting
16:26:43 (1899): No heartbeat from core client for 30 sec - exiting
16:26:44 (1899): No heartbeat from core client for 30 sec - exiting
16:26:45 (1899): No heartbeat from core client for 30 sec - exiting
16:26:46 (1899): No heartbeat from core client for 30 sec - exiting
16:26:47 (1899): No heartbeat from core client for 30 sec - exiting
16:26:48 (1899): No heartbeat from core client for 30 sec - exiting
16:26:49 (1899): No heartbeat from core client for 30 sec - exiting
16:26:50 (1899): No heartbeat from core client for 30 sec - exiting
16:26:51 (1899): No heartbeat from core client for 30 sec - exiting
16:26:52 (1899): No heartbeat from core client for 30 sec - exiting
16:26:53 (1899): No heartbeat from core client for 30 sec - exiting
16:26:54 (1899): No heartbeat from core client for 30 sec - exiting
16:26:55 (1899): No heartbeat from core client for 30 sec - exiting
16:26:56 (1899): No heartbeat from core client for 30 sec - exiting
16:30:53 (1899): No heartbeat from core client for 30 sec - exiting
16:30:54 (1899): No heartbeat from core client for 30 sec - exiting
16:30:55 (1899): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08b1b390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x097dd390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0904b390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09f10390 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
07 Oct 2013 22:12:26 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 259,200 355,495 1.3715
06 Oct 2013 18:55:28 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 233,280 321,521 1.3783
05 Oct 2013 18:37:29 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 207,360 285,817 1.3784
05 Oct 2013 02:16:13 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 181,440 249,824 1.3769
04 Oct 2013 10:52:47 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 155,520 213,929 1.3756
03 Oct 2013 22:05:51 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 129,600 179,016 1.3813
03 Oct 2013 07:15:28 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 103,680 143,219 1.3814
02 Oct 2013 17:48:16 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 77,760 106,625 1.3712
01 Oct 2013 23:14:04 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 51,840 70,811 1.3660
01 Oct 2013 08:05:26 1295603 16050729 hadcm3n_48b7_1980_40_008340403_4 25,920 35,848 1.3830


©2024 cpdn.org