Task 15523373

Name	hadcm3n_zd0s_1880_40_008200973_4
Workunit	8356097
Created	5 Jan 2013, 6:44:59 UTC
Sent	5 Jan 2013, 6:45:11 UTC
Report deadline	6 Apr 2013, 14:12:22 UTC
Received	13 May 2016, 17:57:52 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	742062
Run time	40 days 14 hours 25 min 53 sec
CPU time	40 days 14 hours 25 min 53 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>5.10.21</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:00:34 (15484): No heartbeat from core client for 30 sec - exiting 20:00:35 (15484): No heartbeat from core client for 30 sec - exiting 20:00:36 (15484): No heartbeat from core client for 30 sec - exiting 20:00:37 (15484): No heartbeat from core client for 30 sec - exiting 20:00:38 (15484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/zd0sko.pj95c10 is not a valid UM file. Error converting file to netcdf: dataout/zd0sko.pj95c10 Error: Input file: dataout/zd0sko.pi95c10 is not a valid UM file. Error converting file to netcdf: dataout/zd0sko.pi95c10 Error: Input file: dataout/zd0sko.pf95c10 is not a valid UM file. Error converting file to netcdf: dataout/zd0sko.pf95c10 Error: Input file: dataout/zd0ska.ph95c10 is not a valid UM file. Error converting file to netcdf: dataout/zd0ska.ph95c10 Error: Input file: dataout/zd0ska.pg95c10 is not a valid UM file. Error converting file to netcdf: dataout/zd0ska.pg95c10 Error: Input file: dataout/zd0ska.pe95c10 is not a valid UM file. Error converting file to netcdf: dataout/zd0ska.pe95c10 Error: Input file: dataout/zd0ska.pd95c10 is not a valid UM file. Error converting file to netcdf: dataout/zd0ska.pd95c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:37:34 (603): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:37:38 (603): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08191030 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:3096: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 07:22:14 (528): No heartbeat from core client for 30 sec - exiting 07:22:18 (528): No heartbeat from core client for 30 sec - exiting 07:22:19 (528): No heartbeat from core client for 30 sec - exiting 07:22:20 (528): No heartbeat from core client for 30 sec - exiting 18:06:33 (528): No heartbeat from core client for 30 sec - exiting 18:06:34 (528): No heartbeat from core client for 30 sec - exiting 18:06:35 (528): No heartbeat from core client for 30 sec - exiting 18:06:36 (528): No heartbeat from core client for 30 sec - exiting 18:06:37 (528): No heartbeat from core client for 30 sec - exiting 18:06:38 (528): No heartbeat from core client for 30 sec - exiting 18:06:39 (528): No heartbeat from core client for 30 sec - exiting 18:06:40 (528): No heartbeat from core client for 30 sec - exiting 18:06:41 (528): No heartbeat from core client for 30 sec - exiting 18:06:42 (528): No heartbeat from core client for 30 sec - exiting 18:06:43 (528): No heartbeat from core client for 30 sec - exiting 18:06:44 (528): No heartbeat from core client for 30 sec - exiting 18:06:45 (528): No heartbeat from core client for 30 sec - exiting 18:06:46 (528): No heartbeat from core client for 30 sec - exiting 18:06:47 (528): No heartbeat from core client for 30 sec - exiting 18:06:48 (528): No heartbeat from core client for 30 sec - exiting 18:06:49 (528): No heartbeat from core client for 30 sec - exiting 18:06:50 (528): No heartbeat from core client for 30 sec - exiting 18:06:51 (528): No heartbeat from core client for 30 sec - exiting 18:06:52 (528): No heartbeat from core client for 30 sec - exiting 18:06:53 (528): No heartbeat from core client for 30 sec - exiting 18:06:55 (528): No heartbeat from core client for 30 sec - exiting 18:06:56 (528): No heartbeat from core client for 30 sec - exiting 18:06:57 (528): No heartbeat from core client for 30 sec - exiting 18:06:58 (528): No heartbeat from core client for 30 sec - exiting 18:06:59 (528): No heartbeat from core client for 30 sec - exiting 18:07:00 (528): No heartbeat from core client for 30 sec - exiting 18:07:01 (528): No heartbeat from core client for 30 sec - exiting 18:07:02 (528): No heartbeat from core client for 30 sec - exiting 18:07:03 (528): No heartbeat from core client for 30 sec - exiting 18:07:04 (528): No heartbeat from core client for 30 sec - exiting 18:07:05 (528): No heartbeat from core client for 30 sec - exiting 18:07:06 (528): No heartbeat from core client for 30 sec - exiting 18:07:07 (528): No heartbeat from core client for 30 sec - exiting 18:07:08 (528): No heartbeat from core client for 30 sec - exiting 18:07:09 (528): No heartbeat from core client for 30 sec - exiting 18:07:10 (528): No heartbeat from core client for 30 sec - exiting 18:07:11 (528): No heartbeat from core client for 30 sec - exiting 18:07:12 (528): No heartbeat from core client for 30 sec - exiting 18:07:13 (528): No heartbeat from core client for 30 sec - exiting 18:07:14 (528): No heartbeat from core client for 30 sec - exiting 18:07:15 (528): No heartbeat from core client for 30 sec - exiting 18:07:16 (528): No heartbeat from core client for 30 sec - exiting 18:07:17 (528): No heartbeat from core client for 30 sec - exiting 18:07:18 (528): No heartbeat from core client for 30 sec - exiting 18:07:19 (528): No heartbeat from core client for 30 sec - exiting 18:07:20 (528): No heartbeat from core client for 30 sec - exiting 18:07:21 (528): No heartbeat from core client for 30 sec - exiting 18:07:22 (528): No heartbeat from core client for 30 sec - exiting 18:07:23 (528): No heartbeat from core client for 30 sec - exiting 18:07:24 (528): No heartbeat from core client for 30 sec - exiting 18:07:25 (528): No heartbeat from core client for 30 sec - exiting 18:07:26 (528): No heartbeat from core client for 30 sec - exiting 18:07:27 (528): No heartbeat from core client for 30 sec - exiting 18:07:28 (528): No heartbeat from core client for 30 sec - exiting 18:07:29 (528): No heartbeat from core client for 30 sec - exiting 18:07:30 (528): No heartbeat from core client for 30 sec - exiting 18:07:31 (528): No heartbeat from core client for 30 sec - exiting 18:07:32 (528): No heartbeat from core client for 30 sec - exiting 18:07:33 (528): No heartbeat from core client for 30 sec - exiting 18:07:34 (528): No heartbeat from core client for 30 sec - exiting 18:07:35 (528): No heartbeat from core client for 30 sec - exiting 18:07:36 (528): No heartbeat from core client for 30 sec - exiting 18:07:37 (528): No heartbeat from core client for 30 sec - exiting 18:07:38 (528): No heartbeat from core client for 30 sec - exiting 18:07:39 (528): No heartbeat from core client for 30 sec - exiting 18:07:40 (528): No heartbeat from core client for 30 sec - exiting 18:07:41 (528): No heartbeat from core client for 30 sec - exiting 18:07:42 (528): No heartbeat from core client for 30 sec - exiting 18:07:43 (528): No heartbeat from core client for 30 sec - exiting 18:07:44 (528): No heartbeat from core client for 30 sec - exiting 18:07:45 (528): No heartbeat from core client for 30 sec - exiting 18:07:46 (528): No heartbeat from core client for 30 sec - exiting 18:07:47 (528): No heartbeat from core client for 30 sec - exiting 18:07:48 (528): No heartbeat from core client for 30 sec - exiting 18:07:49 (528): No heartbeat from core client for 30 sec - exiting 18:07:50 (528): No heartbeat from core client for 30 sec - exiting 18:07:51 (528): No heartbeat from core client for 30 sec - exiting * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x08e56830 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x0a17e830 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 18:40:35 (1361): No heartbeat from core client for 30 sec - exiting 18:40:36 (1361): No heartbeat from core client for 30 sec - exiting 18:40:37 (1361): No heartbeat from core client for 30 sec - exiting 18:40:38 (1361): No heartbeat from core client for 30 sec - exiting 18:40:39 (1361): No heartbeat from core client for 30 sec - exiting 18:40:40 (1361): No heartbeat from core client for 30 sec - exiting 18:40:41 (1361): No heartbeat from core client for 30 sec - exiting 18:40:42 (1361): No heartbeat from core client for 30 sec - exiting 18:40:43 (1361): No heartbeat from core client for 30 sec - exiting 18:40:44 (1361): No heartbeat from core client for 30 sec - exiting 18:40:45 (1361): No heartbeat from core client for 30 sec - exiting 18:40:46 (1361): No heartbeat from core client for 30 sec - exiting 18:40:47 (1361): No heartbeat from core client for 30 sec - exiting 18:40:48 (1361): No heartbeat from core client for 30 sec - exiting 18:40:49 (1361): No heartbeat from core client for 30 sec - exiting 18:40:50 (1361): No heartbeat from core client for 30 sec - exiting 18:40:51 (1361): No heartbeat from core client for 30 sec - exiting 18:40:52 (1361): No heartbeat from core client for 30 sec - exiting 18:40:53 (1361): No heartbeat from core client for 30 sec - exiting 18:40:54 (1361): No heartbeat from core client for 30 sec - exiting 18:40:55 (1361): No heartbeat from core client for 30 sec - exiting 18:40:56 (1361): No heartbeat from core client for 30 sec - exiting 18:40:57 (1361): No heartbeat from core client for 30 sec - exiting 18:40:58 (1361): No heartbeat from core client for 30 sec - exiting 18:40:59 (1361): No heartbeat from core client for 30 sec - exiting 18:41:00 (1361): No heartbeat from core client for 30 sec - exiting 18:41:01 (1361): No heartbeat from core client for 30 sec - exiting 18:41:02 (1361): No heartbeat from core client for 30 sec - exiting 18:41:03 (1361): No heartbeat from core client for 30 sec - exiting 18:41:04 (1361): No heartbeat from core client for 30 sec - exiting 18:41:05 (1361): No heartbeat from core client for 30 sec - exiting 18:41:06 (1361): No heartbeat from core client for 30 sec - exiting 18:41:38 (1361): No heartbeat from core client for 30 sec - exiting 18:41:39 (1361): No heartbeat from core client for 30 sec - exiting 18:41:40 (1361): No heartbeat from core client for 30 sec - exiting 18:41:41 (1361): No heartbeat from core client for 30 sec - exiting 18:41:42 (1361): No heartbeat from core client for 30 sec - exiting 18:41:43 (1361): No heartbeat from core client for 30 sec - exiting 18:41:44 (1361): No heartbeat from core client for 30 sec - exiting 18:41:45 (1361): No heartbeat from core client for 30 sec - exiting 18:41:46 (1361): No heartbeat from core client for 30 sec - exiting 18:41:47 (1361): No heartbeat from core client for 30 sec - exiting 18:41:48 (1361): No heartbeat from core client for 30 sec - exiting 18:41:49 (1361): No heartbeat from core client for 30 sec - exiting 18:41:50 (1361): No heartbeat from core client for 30 sec - exiting 18:41:51 (1361): No heartbeat from core client for 30 sec - exiting 18:41:52 (1361): No heartbeat from core client for 30 sec - exiting 18:41:53 (1361): No heartbeat from core client for 30 sec - exiting 18:41:54 (1361): No heartbeat from core client for 30 sec - exiting 18:41:55 (1361): No heartbeat from core client for 30 sec - exiting 18:41:56 (1361): No heartbeat from core client for 30 sec - exiting 18:41:57 (1361): No heartbeat from core client for 30 sec - exiting 18:41:58 (1361): No heartbeat from core client for 30 sec - exiting 18:41:59 (1361): No heartbeat from core client for 30 sec - exiting 18:42:00 (1361): No heartbeat from core client for 30 sec - exiting 18:42:01 (1361): No heartbeat from core client for 30 sec - exiting 18:42:02 (1361): No heartbeat from core client for 30 sec - exiting 18:42:03 (1361): No heartbeat from core client for 30 sec - exiting 18:42:04 (1361): No heartbeat from core client for 30 sec - exiting 18:42:05 (1361): No heartbeat from core client for 30 sec - exiting 18:42:06 (1361): No heartbeat from core client for 30 sec - exiting 18:42:07 (1361): No heartbeat from core client for 30 sec - exiting 18:42:08 (1361): No heartbeat from core client for 30 sec - exiting 18:42:09 (1361): No heartbeat from core client for 30 sec - exiting 18:42:10 (1361): No heartbeat from core client for 30 sec - exiting 18:42:11 (1361): No heartbeat from core client for 30 sec - exiting 18:42:12 (1361): No heartbeat from core client for 30 sec - exiting 18:42:13 (1361): No heartbeat from core client for 30 sec - exiting 18:42:14 (1361): No heartbeat from core client for 30 sec - exiting 18:42:15 (1361): No heartbeat from core client for 30 sec - exiting 18:42:16 (1361): No heartbeat from core client for 30 sec - exiting 18:42:17 (1361): No heartbeat from core client for 30 sec - exiting 18:42:18 (1361): No heartbeat from core client for 30 sec - exiting 18:42:19 (1361): No heartbeat from core client for 30 sec - exiting 18:42:20 (1361): No heartbeat from core client for 30 sec - exiting 18:42:21 (1361): No heartbeat from core client for 30 sec - exiting 18:42:22 (1361): No heartbeat from core client for 30 sec - exiting 18:42:23 (1361): No heartbeat from core client for 30 sec - exiting 18:42:24 (1361): No heartbeat from core client for 30 sec - exiting 18:42:25 (1361): No heartbeat from core client for 30 sec - exiting 18:42:26 (1361): No heartbeat from core client for 30 sec - exiting 18:42:27 (1361): No heartbeat from core client for 30 sec - exiting 18:42:28 (1361): No heartbeat from core client for 30 sec - exiting 18:42:29 (1361): No heartbeat from core client for 30 sec - exiting 18:42:30 (1361): No heartbeat from core client for 30 sec - exiting 18:42:31 (1361): No heartbeat from core client for 30 sec - exiting 18:42:32 (1361): No heartbeat from core client for 30 sec - exiting 18:42:33 (1361): No heartbeat from core client for 30 sec - exiting 18:42:34 (1361): No heartbeat from core client for 30 sec - exiting 18:42:35 (1361): No heartbeat from core client for 30 sec - exiting 18:42:36 (1361): No heartbeat from core client for 30 sec - exiting 18:42:37 (1361): No heartbeat from core client for 30 sec - exiting 18:42:38 (1361): No heartbeat from core client for 30 sec - exiting 18:42:39 (1361): No heartbeat from core client for 30 sec - exiting 18:42:40 (1361): No heartbeat from core client for 30 sec - exiting 18:42:41 (1361): No heartbeat from core client for 30 sec - exiting 18:42:42 (1361): No heartbeat from core client for 30 sec - exiting 18:42:43 (1361): No heartbeat from core client for 30 sec - exiting 18:42:44 (1361): No heartbeat from core client for 30 sec - exiting 18:42:45 (1361): No heartbeat from core client for 30 sec - exiting 18:42:46 (1361): No heartbeat from core client for 30 sec - exiting 18:42:47 (1361): No heartbeat from core client for 30 sec - exiting 18:42:48 (1361): No heartbeat from core client for 30 sec - exiting 18:42:49 (1361): No heartbeat from core client for 30 sec - exiting 18:42:50 (1361): No heartbeat from core client for 30 sec - exiting 18:42:51 (1361): No heartbeat from core client for 30 sec - exiting 18:42:52 (1361): No heartbeat from core client for 30 sec - exiting 18:42:53 (1361): No heartbeat from core client for 30 sec - exiting 18:42:54 (1361): No heartbeat from core client for 30 sec - exiting 18:42:55 (1361): No heartbeat from core client for 30 sec - exiting 18:42:56 (1361): No heartbeat from core client for 30 sec - exiting 18:42:57 (1361): No heartbeat from core client for 30 sec - exiting 18:42:58 (1361): No heartbeat from core client for 30 sec - exiting 18:42:59 (1361): No heartbeat from core client for 30 sec - exiting 18:43:00 (1361): No heartbeat from core client for 30 sec - exiting 18:43:32 (1361): No heartbeat from core client for 30 sec - exiting 18:43:33 (1361): No heartbeat from core client for 30 sec - exiting 18:43:34 (1361): No heartbeat from core client for 30 sec - exiting 18:43:35 (1361): No heartbeat from core client for 30 sec - exiting 18:43:36 (1361): No heartbeat from core client for 30 sec - exiting 18:43:37 (1361): No heartbeat from core client for 30 sec - exiting 18:43:38 (1361): No heartbeat from core client for 30 sec - exiting 18:43:39 (1361): No heartbeat from core client for 30 sec - exiting 18:43:40 (1361): No heartbeat from core client for 30 sec - exiting 18:43:41 (1361): No heartbeat from core client for 30 sec - exiting 18:43:42 (1361): No heartbeat from core client for 30 sec - exiting 18:43:43 (1361): No heartbeat from core client for 30 sec - exiting 18:43:44 (1361): No heartbeat from core client for 30 sec - exiting 18:43:45 (1361): No heartbeat from core client for 30 sec - exiting 18:43:46 (1361): No heartbeat from core client for 30 sec - exiting 18:43:47 (1361): No heartbeat from core client for 30 sec - exiting 18:43:48 (1361): No heartbeat from core client for 30 sec - exiting 18:43:49 (1361): No heartbeat from core client for 30 sec - exiting 18:43:50 (1361): No heartbeat from core client for 30 sec - exiting 18:43:51 (1361): No heartbeat from core client for 30 sec - exiting 18:43:52 (1361): No heartbeat from core client for 30 sec - exiting 18:43:53 (1361): No heartbeat from core client for 30 sec - exiting 18:43:54 (1361): No heartbeat from core client for 30 sec - exiting 18:43:55 (1361): No heartbeat from core client for 30 sec - exiting 18:43:56 (1361): No heartbeat from core client for 30 sec - exiting 18:43:57 (1361): No heartbeat from core client for 30 sec - exiting 18:43:58 (1361): No heartbeat from core client for 30 sec - exiting * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x08905830 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x08b9d830 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x09120830 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x09c8e830 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x0a013830 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 12:13:19 (1062): No heartbeat from core client for 30 sec - exiting 12:13:20 (1062): No heartbeat from core client for 30 sec - exiting 12:13:21 (1062): No heartbeat from core client for 30 sec - exiting 12:13:22 (1062): No heartbeat from core client for 30 sec - exiting 12:13:23 (1062): No heartbeat from core client for 30 sec - exiting 12:13:24 (1062): No heartbeat from core client for 30 sec - exiting 12:13:25 (1062): No heartbeat from core client for 30 sec - exiting 12:13:26 (1062): No heartbeat from core client for 30 sec - exiting 12:13:27 (1062): No heartbeat from core client for 30 sec - exiting 12:13:28 (1062): No heartbeat from core client for 30 sec - exiting 12:13:29 (1062): No heartbeat from core client for 30 sec - exiting 12:13:30 (1062): No heartbeat from core client for 30 sec - exiting 12:13:31 (1062): No heartbeat from core client for 30 sec - exiting 12:13:32 (1062): No heartbeat from core client for 30 sec - exiting 12:13:33 (1062): No heartbeat from core client for 30 sec - exiting 12:13:34 (1062): No heartbeat from core client for 30 sec - exiting 12:13:35 (1062): No heartbeat from core client for 30 sec - exiting 12:13:36 (1062): No heartbeat from core client for 30 sec - exiting 12:13:37 (1062): No heartbeat from core client for 30 sec - exiting 12:13:38 (1062): No heartbeat from core client for 30 sec - exiting 12:13:39 (1062): No heartbeat from core client for 30 sec - exiting 12:13:40 (1062): No heartbeat from core client for 30 sec - exiting 12:13:41 (1062): No heartbeat from core client for 30 sec - exiting 12:13:42 (1062): No heartbeat from core client for 30 sec - exiting 12:13:43 (1062): No heartbeat from core client for 30 sec - exiting 12:13:44 (1062): No heartbeat from core client for 30 sec - exiting 12:13:45 (1062): No heartbeat from core client for 30 sec - exiting 12:13:46 (1062): No heartbeat from core client for 30 sec - exiting 12:13:47 (1062): No heartbeat from core client for 30 sec - exiting 12:13:48 (1062): No heartbeat from core client for 30 sec - exiting 12:13:49 (1062): No heartbeat from core client for 30 sec - exiting 12:13:50 (1062): No heartbeat from core client for 30 sec - exiting * Error in `hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x09c5b230 * ======= Backtrace: ========= /lib/libc.so.6(+0x68478)[0xf7418478] /lib/libc.so.6(+0x6e787)[0xf741e787] /lib/libc.so.6(+0x6ef41)[0xf741ef41] /usr/lib/libstdc++.so.6(_ZdlPv+0x18)[0xf764e018] /usr/lib/libstdc++.so.6(_ZdaPv+0x18)[0xf764e078] hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] hadcm3n_6.07_i686-pc-linux-gnu[0x80510cc] hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf7)[0xf73c8527] hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:11 58769937 /archive/dpc/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rwxp 0009b000 08:11 58769937 /archive/dpc/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rwxp 00000000 00:00 0 09c00000-09c72000 rwxp 00000000 00:00 0 [heap] f6e00000-f6e21000 rwxp 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6f32000-f73ad000 rwxs 00000000 08:11 58770538 /archive/dpc/BOINC/slots/0/136605 f73ad000-f73b0000 rwxp 00000000 00:00 0 f73b0000-f755f000 r-xp 00000000 08:06 131392 /lib/libc-2.23.so f755f000-f7560000 ---p 001af000 08:06 131392 /lib/libc-2.23.so f7560000-f7562000 r-xp 001af000 08:06 131392 /lib/libc-2.23.so f7562000-f7563000 rwxp 001b1000 08:06 131392 /lib/libc-2.23.so f7563000-f7566000 rwxp 00000000 00:00 0 f7566000-f757f000 r-xp 00000000 08:06 132147 /lib/libgcc_s.so.1 f757f000-f7580000 ---p 00019000 08:06 132147 /lib/libgcc_s.so.1 f7580000-f7581000 r-xp 00019000 08:06 132147 /lib/libgcc_s.so.1 f7581000-f7582000 rwxp 0001a000 08:06 132147 /lib/libgcc_s.so.1 f7582000-f7583000 rwxp 00000000 00:00 0 f7583000-f75d6000 r-xp 00000000 08:06 134604 /lib/libm-2.23.so f75d6000-f75d7000 r-xp 00052000 08:06 134604 /lib/libm-2.23.so f75d7000-f75d8000 rwxp 00053000 08:06 134604 /lib/libm-2.23.so f75d8000-f7745000 r-xp 00000000 08:06 1647061 /usr/lib/libstdc++.so.6.0.21 f7745000-f7746000 ---p 0016d000 08:06 1647061 /usr/lib/libstdc++.so.6.0.21 f7746000-f774b000 r-xp 0016d000 08:06 1647061 /usr/lib/libstdc++.so.6.0.21 f774b000-f774c000 rwxp 00172000 08:06 1647061 /usr/lib/libstdc++.so.6.0.21 f774c000-f774f000 rwxp 00000000 00:00 0 f774f000-f7752000 r-xp 00000000 08:06 134602 /lib/libdl-2.23.so f7752000-f7753000 r-xp 00002000 08:06 134602 /lib/libdl-2.23.so f7753000-f7754000 rwxp 00003000 08:06 134602 /lib/libdl-2.23.so f7754000-f776d000 r-xp 00000000 08:06 131375 /lib/libpthread-2.23.so f776d000-f776e000 r-xp 00018000 08:06 131375 /lib/libpthread-2.23.so f776e000-f776f000 rwxp 00019000 08:06 131375 /lib/libpthread-2.23.so f776f000-f7771000 rwxp 00000000 00:00 0 f77b6000-f77b7000 rwxp 00000000 00:00 0 f77b7000-f77b8000 ---p 00000000 00:00 0 f77b8000-f77bb000 rwxp 00000000 00:00 0 [stack:1457] f77bb000-f77bd000 rwxs 00000000 00:05 0 /SYSV0111c48e (deleted) f77bd000-f77be000 rwxp 00000000 00:00 0 f77be000-f77c0000 r--p 00000000 00:00 0 [vvar] f77c0000-f77c2000 r-xp 00000000 00:00 0 [vdso] f77c2000-f77e5000 r-xp 00000000 08:06 131359 /lib/ld-2.23.so f77e5000-f77e6000 r-xp 00022000 08:06 131359 /lib/ld-2.23.so f77e6000-f77e7000 rwxp 00023000 08:06 131359 /lib/ld-2.23.so ff950000-ff9c0000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77c0d10] linux-gate.so.1(__kernel_vsyscall+0x5)[0xf77c0d35] /lib/libc.so.6(gsignal+0x39)[0xf73dd3f9] /lib/libc.so.6(abort+0x121)[0xf73de921] /lib/libc.so.6(+0x6847d)[0xf741847d] /lib/libc.so.6(+0x6e787)[0xf741e787] /lib/libc.so.6(+0x6ef41)[0xf741ef41] /usr/lib/libstdc++.so.6(_ZdlPv+0x18)[0xf764e018] /usr/lib/libstdc++.so.6(_ZdaPv+0x18)[0xf764e078] hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] hadcm3n_6.07_i686-pc-linux-gnu[0x80510cc] hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xf7)[0xf73c8527] hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Mar 2013 22:35:37	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	1,036,800	3,506,543	3.3821
26 Mar 2013 22:51:55	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	1,010,880	3,420,742	3.3839
25 Mar 2013 07:57:02	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	984,960	3,336,480	3.3874
22 Mar 2013 21:47:34	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	959,040	3,251,164	3.3900
21 Mar 2013 15:35:35	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	933,120	3,166,258	3.3932
17 Mar 2013 01:02:17	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	907,200	3,081,357	3.3966
15 Mar 2013 06:02:45	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	881,280	2,996,106	3.3997
13 Mar 2013 20:44:23	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	855,360	2,910,997	3.4032
12 Mar 2013 00:29:04	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	829,440	2,825,419	3.4064
06 Mar 2013 08:45:49	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	803,520	2,738,264	3.4078
05 Mar 2013 07:45:36	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	777,600	2,650,616	3.4087
04 Mar 2013 06:37:54	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	751,680	2,563,165	3.4099
03 Mar 2013 05:40:10	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	725,760	2,475,724	3.4112
02 Mar 2013 04:16:14	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	699,840	2,387,833	3.4120
01 Mar 2013 02:46:02	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	673,920	2,299,646	3.4123
28 Feb 2013 01:05:41	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	648,000	2,210,135	3.4107
26 Feb 2013 23:44:35	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	622,080	2,121,237	3.4099
25 Feb 2013 22:06:30	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	596,160	2,032,479	3.4093
24 Feb 2013 20:32:22	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	570,240	1,943,962	3.4090
23 Feb 2013 19:15:17	742062	15523373	hadcm3n_zd0s_1880_40_008200973_4	544,320	1,856,346	3.4104