From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 316CB2B7DA for ; Sun, 28 Apr 2019 12:56:31 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d0AmeQzfi5_5 for ; Sun, 28 Apr 2019 12:56:31 -0400 (EDT) Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id C0A892B6FE for ; Sun, 28 Apr 2019 12:56:30 -0400 (EDT) From: Vladislav Shpilevoy Subject: [tarantool-patches] [PATCH 1/3] small: introduce small/static Date: Sun, 28 Apr 2019 19:56:25 +0300 Message-Id: <7a0ec60e4d19756f3a6ab1588b1b1dda860c1749.1556470563.git.v.shpilevoy@tarantool.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: kostja@tarantool.org Before the patch Tarantool had a thread- and C-file- local array of 4 static buffers, each 1028 bytes. It provided an API tt_static_buf() allowing to return them one by one in a cycle. Firstly, it consumed totally 200Kb of BSS memory in summary over all C-files using these buffers. Obviously, it was a bug and was not made intentionally. The buffers were supposed to be a one process-global array. Secondly, even if the bug above had been fixed somehow, sometimes it would have been needed to obtain a bit bigger buffer. For example, to store a UDP packet - ~1.5Kb. This commit replaces these 4 buffers with small/ static allocator which does basically the same, but in more granulated and manoeuvrable way. This commit frees ~188Kb of BSS section. A main motivation for this commit is a wish to use a single global out-of-stack buffer to read UDP packets into it in the SWIM library, and on the other hand do not pad out BSS section with a new SWIM-special static buffer. Now SWIM uses stack for this and in the incoming cryptography SWIM component it will need more. --- src/box/call.c | 1 + src/box/identifier.c | 2 +- src/box/identifier.h | 1 - src/box/iproto.cc | 1 + src/box/opt_def.c | 1 + src/box/relay.cc | 2 +- src/box/session.cc | 1 + src/box/space_def.c | 1 + src/box/space_def.h | 2 +- src/box/sql/sqlLimit.h | 2 +- src/box/tuple.h | 1 + src/box/tuple_format.c | 1 + src/box/user.cc | 1 + src/box/vclock.c | 2 +- src/box/vy_log.c | 2 +- src/box/xrow.c | 2 +- src/lib/coll/coll.c | 2 +- src/lib/core/fio.c | 2 +- src/lib/core/say.c | 1 + src/lib/core/sio.c | 9 +-- src/lib/core/tt_static.h | 118 ++++++++++++++++++++++++++++++++++ src/lib/small | 2 +- src/lib/swim/swim_io.h | 2 +- src/lib/swim/swim_proto.h | 2 +- src/lib/swim/swim_transport.h | 1 - src/lib/uuid/tt_uuid.c | 2 +- src/lua/utf8.c | 1 + src/trivia/util.h | 53 --------------- 28 files changed, 146 insertions(+), 72 deletions(-) create mode 100644 src/lib/core/tt_static.h diff --git a/src/box/call.c b/src/box/call.c index b9750c5f3..56da53fb3 100644 --- a/src/box/call.c +++ b/src/box/call.c @@ -41,6 +41,7 @@ #include "iproto_constants.h" #include "rmean.h" #include "small/obuf.h" +#include "tt_static.h" /** * Find a function by name and check "EXECUTE" permissions. diff --git a/src/box/identifier.c b/src/box/identifier.c index 252d53618..b1c56bdc3 100644 --- a/src/box/identifier.c +++ b/src/box/identifier.c @@ -32,7 +32,7 @@ #include "say.h" #include "diag.h" - +#include "tt_static.h" #include #include diff --git a/src/box/identifier.h b/src/box/identifier.h index f34380d62..a0ed6c10e 100644 --- a/src/box/identifier.h +++ b/src/box/identifier.h @@ -30,7 +30,6 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ -#include "trivia/util.h" #include #include "error.h" diff --git a/src/box/iproto.cc b/src/box/iproto.cc index 10b6a6792..b9d2004f0 100644 --- a/src/box/iproto.cc +++ b/src/box/iproto.cc @@ -62,6 +62,7 @@ #include "rmean.h" #include "execute.h" #include "errinj.h" +#include "tt_static.h" enum { IPROTO_SALT_SIZE = 32, diff --git a/src/box/opt_def.c b/src/box/opt_def.c index c78021440..e2820853b 100644 --- a/src/box/opt_def.c +++ b/src/box/opt_def.c @@ -35,6 +35,7 @@ #include "small/region.h" #include "error.h" #include "diag.h" +#include "tt_static.h" const char *opt_type_strs[] = { /* [OPT_BOOL] = */ "boolean", diff --git a/src/box/relay.cc b/src/box/relay.cc index e3d7f0ed1..906bf8efd 100644 --- a/src/box/relay.cc +++ b/src/box/relay.cc @@ -31,7 +31,7 @@ #include "relay.h" #include "trivia/config.h" -#include "trivia/util.h" +#include "tt_static.h" #include "scoped_guard.h" #include "cbus.h" #include "cfg.h" diff --git a/src/box/session.cc b/src/box/session.cc index 70822e8fd..4bb13d031 100644 --- a/src/box/session.cc +++ b/src/box/session.cc @@ -35,6 +35,7 @@ #include "trigger.h" #include "user.h" #include "error.h" +#include "tt_static.h" const char *session_type_strs[] = { "background", diff --git a/src/box/space_def.c b/src/box/space_def.c index d0864cc72..d825def75 100644 --- a/src/box/space_def.c +++ b/src/box/space_def.c @@ -34,6 +34,7 @@ #include "error.h" #include "sql.h" #include "msgpuck.h" +#include "tt_static.h" /** * Make checks from msgpack. diff --git a/src/box/space_def.h b/src/box/space_def.h index 52ff56764..c6639a8d8 100644 --- a/src/box/space_def.h +++ b/src/box/space_def.h @@ -30,9 +30,9 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ -#include "trivia/util.h" #include "tuple_dictionary.h" #include "schema_def.h" +#include #include #include diff --git a/src/box/sql/sqlLimit.h b/src/box/sql/sqlLimit.h index 76e0d9c53..53dbe1505 100644 --- a/src/box/sql/sqlLimit.h +++ b/src/box/sql/sqlLimit.h @@ -35,7 +35,7 @@ * * This file defines various limits of what sql can process. */ -#include "trivia/util.h" +#include "tt_static.h" enum { /* diff --git a/src/box/tuple.h b/src/box/tuple.h index eed8e1a34..a496bf6f0 100644 --- a/src/box/tuple.h +++ b/src/box/tuple.h @@ -35,6 +35,7 @@ #include "diag.h" #include "error.h" #include "uuid/tt_uuid.h" /* tuple_field_uuid */ +#include "tt_static.h" #include "tuple_format.h" #if defined(__cplusplus) diff --git a/src/box/tuple_format.c b/src/box/tuple_format.c index 804a678a1..9f742d900 100644 --- a/src/box/tuple_format.c +++ b/src/box/tuple_format.c @@ -33,6 +33,7 @@ #include "json/json.h" #include "tuple_format.h" #include "coll_id_cache.h" +#include "tt_static.h" #include "third_party/PMurHash.h" diff --git a/src/box/user.cc b/src/box/user.cc index 403d6d6c4..48bdf18ee 100644 --- a/src/box/user.cc +++ b/src/box/user.cc @@ -38,6 +38,7 @@ #include "session.h" #include "scoped_guard.h" #include "sequence.h" +#include "tt_static.h" struct universe universe; static struct user users[BOX_USER_MAX]; diff --git a/src/box/vclock.c b/src/box/vclock.c index cea67017c..90ae27591 100644 --- a/src/box/vclock.c +++ b/src/box/vclock.c @@ -35,7 +35,7 @@ #include #include "diag.h" -#include "trivia/util.h" +#include "tt_static.h" int64_t vclock_follow(struct vclock *vclock, uint32_t replica_id, int64_t lsn) diff --git a/src/box/vy_log.c b/src/box/vy_log.c index be97cdbbe..de39b746f 100644 --- a/src/box/vy_log.c +++ b/src/box/vy_log.c @@ -57,7 +57,7 @@ #include "replication.h" /* INSTANCE_UUID */ #include "salad/stailq.h" #include "say.h" -#include "trivia/util.h" +#include "tt_static.h" #include "wal.h" #include "vclock.h" #include "xlog.h" diff --git a/src/box/xrow.c b/src/box/xrow.c index aaed84e38..fef4f2dff 100644 --- a/src/box/xrow.c +++ b/src/box/xrow.c @@ -37,7 +37,7 @@ #include "fiber.h" #include "version.h" - +#include "tt_static.h" #include "error.h" #include "vclock.h" #include "scramble.h" diff --git a/src/lib/coll/coll.c b/src/lib/coll/coll.c index 495eb6d64..98e4d0e64 100644 --- a/src/lib/coll/coll.c +++ b/src/lib/coll/coll.c @@ -35,7 +35,7 @@ #include "assoc.h" #include #include -#include +#include "tt_static.h" struct UCaseMap *icu_ucase_default_map = NULL; diff --git a/src/lib/core/fio.c b/src/lib/core/fio.c index b79d3d058..4388b9a53 100644 --- a/src/lib/core/fio.c +++ b/src/lib/core/fio.c @@ -41,7 +41,7 @@ #include #include -#include +#include "tt_static.h" const char * fio_filename(int fd) diff --git a/src/lib/core/say.c b/src/lib/core/say.c index 3b13b766a..24963cf82 100644 --- a/src/lib/core/say.c +++ b/src/lib/core/say.c @@ -31,6 +31,7 @@ #include "say.h" #include "fiber.h" #include "errinj.h" +#include "tt_static.h" #include #include diff --git a/src/lib/core/sio.c b/src/lib/core/sio.c index 3e1b7e784..64aae97bc 100644 --- a/src/lib/core/sio.c +++ b/src/lib/core/sio.c @@ -38,12 +38,12 @@ #include /* TCP_NODELAY */ #include #include "say.h" -#include "trivia/util.h" +#include "tt_static.h" #include "exception.h" #include "uri/uri.h" #include "errinj.h" -static_assert(TT_STATIC_BUF_LEN > NI_MAXHOST, +static_assert(SMALL_STATIC_SIZE > NI_MAXHOST + NI_MAXSERV, "static buffer should fit host name"); const char * @@ -330,8 +330,9 @@ sio_strfaddr(const struct sockaddr *addr, socklen_t addrlen) NI_NUMERICHOST | NI_NUMERICSERV) != 0) return tt_sprintf("(host):(port)"); - return tt_sprintf(addr->sa_family == AF_INET ? - "%s:%s" : "[%s]:%s", host, serv); + return tt_snprintf(NI_MAXHOST + NI_MAXSERV, + addr->sa_family == AF_INET ? + "%s:%s" : "[%s]:%s", host, serv); } } unreachable(); diff --git a/src/lib/core/tt_static.h b/src/lib/core/tt_static.h new file mode 100644 index 000000000..5621c640c --- /dev/null +++ b/src/lib/core/tt_static.h @@ -0,0 +1,118 @@ +#ifndef TARANTOOL_CORE_STATIC_H_INCLUDED +#define TARANTOOL_CORE_STATIC_H_INCLUDED +/* + * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ +#include "small/static.h" +#include +#include +#include +#include + +#ifndef MIN +#define UNDEF_MIN +#define MIN(a, b) ((a) < (b) ? (a) : (b)) +#endif + +/** Can't be enum - used by preprocessor for static assertions. */ +#define TT_STATIC_BUF_LEN 1024 + +/** + * Return a thread-local statically allocated temporary buffer of + * size @a TT_STATIC_BUF_LEN. + */ +static inline char * +tt_static_buf(void) +{ + return static_alloc(TT_STATIC_BUF_LEN); +} + +/** + * Return a null-terminated string for @a str of length @a len. + */ +static inline const char * +tt_cstr(const char *str, size_t len) +{ + len = MIN(len, SMALL_STATIC_SIZE - 1); + char *buf = static_alloc(len + 1); + memcpy(buf, str, len); + buf[len] = '\0'; + return buf; +} + +/** + * Wrapper around vsnprintf() that prints the result to + * the static buffer. + */ +static inline const char * +tt_vsnprintf(size_t size, const char *format, va_list ap) +{ + char *buf = static_reserve(size); + assert(buf != NULL); + int rc = vsnprintf(buf, size, format, ap); + if (rc >= 0) { + /* +1 for terminating zero. */ + rc = MIN((int) size, rc + 1); + static_alloc(rc); + } + return buf; +} + +/** @copydoc tt_vsnprintf() */ +static inline const char * +tt_sprintf(const char *format, ...) +{ + va_list ap; + va_start(ap, format); + const char *result = tt_vsnprintf(TT_STATIC_BUF_LEN, format, ap); + va_end(ap); + return result; +} + +/** + * The same as tt_sprintf() but allows to specify more precise + * string limits. + */ +static inline const char * +tt_snprintf(size_t size, const char *format, ...) +{ + va_list ap; + va_start(ap, format); + const char *result = tt_vsnprintf(MIN(size, (size_t) SMALL_STATIC_SIZE), + format, ap); + va_end(ap); + return result; +} + +#ifdef UNDEF_MIN +#undef MIN +#endif + +#endif /* TARANTOOL_CORE_STATIC_H_INCLUDED */ diff --git a/src/lib/small b/src/lib/small index cdf7d4a8e..50ba09107 160000 --- a/src/lib/small +++ b/src/lib/small @@ -1 +1 @@ -Subproject commit cdf7d4a8e24ae465298d0ddacaf1aaed1085281e +Subproject commit 50ba091077c074c4eecc1ee63f53d2f4f245bd51 diff --git a/src/lib/swim/swim_io.h b/src/lib/swim/swim_io.h index b17629ba9..42d94fc92 100644 --- a/src/lib/swim/swim_io.h +++ b/src/lib/swim/swim_io.h @@ -30,7 +30,7 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ -#include "trivia/util.h" +#include "tt_static.h" #include "small/rlist.h" #include "salad/stailq.h" #include "swim_transport.h" diff --git a/src/lib/swim/swim_proto.h b/src/lib/swim/swim_proto.h index 81770c01c..beb9bb1fe 100644 --- a/src/lib/swim/swim_proto.h +++ b/src/lib/swim/swim_proto.h @@ -30,7 +30,7 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ -#include "trivia/util.h" +#include "tt_static.h" #include "uuid/tt_uuid.h" #include #include diff --git a/src/lib/swim/swim_transport.h b/src/lib/swim/swim_transport.h index 6f4370087..c4e48cdeb 100644 --- a/src/lib/swim/swim_transport.h +++ b/src/lib/swim/swim_transport.h @@ -30,7 +30,6 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ -#include "trivia/util.h" #include #include diff --git a/src/lib/uuid/tt_uuid.c b/src/lib/uuid/tt_uuid.c index 530aa31a1..1bd2e2cfe 100644 --- a/src/lib/uuid/tt_uuid.c +++ b/src/lib/uuid/tt_uuid.c @@ -33,7 +33,7 @@ #include #include #include -#include +#include "tt_static.h" /* Zeroed by the linker. */ const struct tt_uuid uuid_nil; diff --git a/src/lua/utf8.c b/src/lua/utf8.c index d829b57b4..9c678cad4 100644 --- a/src/lua/utf8.c +++ b/src/lua/utf8.c @@ -35,6 +35,7 @@ #include "lua/utf8.h" #include "diag.h" #include "small/ibuf.h" +#include "tt_static.h" extern struct ibuf *tarantool_lua_ibuf; diff --git a/src/trivia/util.h b/src/trivia/util.h index a3e84e6d5..1e01013db 100644 --- a/src/trivia/util.h +++ b/src/trivia/util.h @@ -479,59 +479,6 @@ int clock_gettime(uint32_t clock_id, struct timespec *tp); #define CLOCK_THREAD_CPUTIME_ID 3 #endif -#define TT_STATIC_BUF_LEN 1028 - -/** - * Return a thread-local statically allocated temporary buffer of size - * \a TT_STATIC_BUF_LEN - */ -static inline char * -tt_static_buf(void) -{ - enum { TT_STATIC_BUFS = 4 }; - static __thread char bufs[TT_STATIC_BUFS][TT_STATIC_BUF_LEN]; - static __thread int bufno = TT_STATIC_BUFS - 1; - - bufno = (bufno + 1) % TT_STATIC_BUFS; - return bufs[bufno]; -} - -/** - * Return a null-terminated string for \a str of length \a len - */ -static inline const char * -tt_cstr(const char *str, size_t len) -{ - char *buf = tt_static_buf(); - len = MIN(len, TT_STATIC_BUF_LEN - 1); - memcpy(buf, str, len); - buf[len] = '\0'; - return buf; -} - -/** - * Wrapper around sprintf() that prints the result to - * the static buffer returned by tt_static_buf(). - */ -static inline const char * -tt_vsprintf(const char *format, va_list ap) -{ - char *buf = tt_static_buf(); - vsnprintf(buf, TT_STATIC_BUF_LEN, format, ap); - return buf; -} - -/** @copydoc tt_vsprintf() */ -static inline const char * -tt_sprintf(const char *format, ...) -{ - va_list ap; - va_start(ap, format); - const char *result = tt_vsprintf(format, ap); - va_end(ap); - return result; -} - /** * Escape special characters in @a data to @a buf */ -- 2.20.1 (Apple Git-117)