From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 5B92A6EC56; Sat, 20 Mar 2021 03:48:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 5B92A6EC56 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1616201336; bh=uo9nG0s8l3kL3spSBcYuwYO3gajnG6TSF1079SkTXWY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=VZUsMK+bLmHW3u3H0iVqZ5af8Lmlo+rihDsn8i88dO/EGCGG6Yjh/uAUothnry2wP w5GcE5JU8OmXjlLnkycSyuPxU7kY2JwwtR6qWRZDqyrW8BjlRMLo0gvibvTA7fbH1e eg0HxGx8J5jREdQoQg2rQPm1oW1KH7Zg6r5uqh8k= Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 917496C7D2 for ; Sat, 20 Mar 2021 03:43:06 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 917496C7D2 Received: by smtpng3.m.smailru.net with esmtpa (envelope-from ) id 1lNPhp-00076U-Ee; Sat, 20 Mar 2021 03:43:06 +0300 To: tarantool-patches@dev.tarantool.org, gorcunov@gmail.com, sergepetrenko@tarantool.org Date: Sat, 20 Mar 2021 01:42:42 +0100 Message-Id: <0523db7b1c18c5a7f6ea5502dc9010fa4c23dd87.1616200860.git.v.shpilevoy@tarantool.org> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD95D6E7CC48CB1F5F1C82687294EF6886BAE1FA542CE3F1C38182A05F538085040F355B0B73DD0630EE0835E8E6072C54DC392AD491B9B697A98E5A3AC0D58C9EC X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7EA3ECAD3D6C46603C2099A533E45F2D0395957E7521B51C2CFCAF695D4D8E9FCEA1F7E6F0F101C6778DA827A17800CE7F12ABE79F2AB44EAEA1F7E6F0F101C67CDEEF6D7F21E0D1D174C73DBBBFC7664D0E9B0F0329DBC1D860B81391F2D02414A6583182E452799389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C07E7E81EEA8A9722B8941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B65FF0BFC5AEE34BE6CC7F00164DA146DA6F5DAA56C3B73B23C77107234E2CFBA567F23339F89546C55F5C1EE8F4F765FCAEB924C2B054B06E75ECD9A6C639B01BBD4B6F7A4D31EC0BC0CAF46E325F83A522CA9DD8327EE4930A3850AC1BE2E735E5B2F3A2B87CD4C8C4224003CC836476C0CAF46E325F83A50BF2EBBBDD9D6B0FECB2555BB02FD5A93B503F486389A921A5CC5B56E945C8DA X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2BBE337FB72E923155C0AF1600DCBC20BD3311767B7455A330DEFC89061B634E8 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8183A4AFAF3EA6BDC44C234C8B12C006B7AD0E9B0F0329DBC1D860B81391F2D0241C05FA0CD7C9EE5DAB1881A6453793CE9C32612AADDFBE061C61BE10805914D3804EBA3D8E7E5B87ABF8C51168CD8EBDB5F0C88D684269EDEDC48ACC2A39D04F89CDFB48F4795C241BDAD6C7F3747799A X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34060C3C6DE316ECE40E951569C04D8B51884B665949C4EEC22A7D19CA08DCE9579EF0F7F6D58E794F1D7E09C32AA3244CAC17507AEAC01AC800A076A345C4DB11A995755A1445935EFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj8xHC0Ak4ylbb2foK22yOvQ== X-Mailru-Sender: 689FA8AB762F73936BC43F508A063822B79DDA91B9A382349EC092231F9FFDC73841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: [Tarantool-patches] [PATCH 05/16] cord_buf: introduce cord_buf API X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" There was a global ibuf object called tarantool_lua_ibuf. It was used in all the places working with Lua which didn't have yields, and where fiber's region could be potentially slower due to not being able to guarantee the allocated memory is contiguous. Yields during the ibuf usage were prohibited because another fiber would take the same ibuf and override its previous content which was still used by another fiber. But it wasn't taken into account that there is Lua GC. It can be invoked from any Lua function in Lua C code, and almost on any line in the Lua scripts. During GC some deleted objects might have GC handlers installed as __gc metamethods. From the handler they could call Tarantool functions, including the ones using the global ibuf. Therefore ibuf could be overridden not only at yields, but almost in any moment. Because with the Lua GC at hand, the multitasking is not strictly "cooperative" anymore. It is necessary to implement ownership for the global buffer. The patch prepares the API for this: the buffer is moved to its own file, and has methods take(), put(), and drop(). Take() is supposed to make the current fiber own the buffer. Put() makes it available again. Drop() does the same but also clears the buffer (frees its memory). The ownership itself is a subject for the next patches. Here only the API is prepared. The patch "hits" performance a little. Previously the get of buffer.IBUF_SHARED cost around 1 ns. Now cord_ibuf_take() + cord_ibuf_put() cost around 5 ns together. The next patches will make it worse, up to 15 ns until #5871 is done. Part of #5632 --- src/box/lua/schema.lua | 47 +++++++++++------ src/box/lua/tuple.c | 28 +++++----- src/box/lua/tuple.lua | 14 +++-- src/exports.h | 4 +- src/lib/core/CMakeLists.txt | 1 + src/lib/core/cord_buf.c | 47 +++++++++++++++++ src/lib/core/cord_buf.h | 45 ++++++++++++++++ src/lua/buffer.lua | 51 ++++++++++++++++++- src/lua/iconv.lua | 8 +-- src/lua/init.c | 3 -- src/lua/msgpack.c | 6 +-- src/lua/msgpackffi.lua | 7 +-- src/lua/socket.lua | 14 +++-- src/lua/swim.lua | 17 ++++--- src/lua/utf8.c | 16 +++--- src/lua/utils.h | 1 - .../test/static-build/exports.test.lua | 4 +- test/unit/luaT_tuple_new.c | 4 -- 18 files changed, 242 insertions(+), 75 deletions(-) create mode 100644 src/lib/core/cord_buf.c create mode 100644 src/lib/core/cord_buf.h diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index fea8640a6..cc5a3451a 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -18,6 +18,8 @@ local tuple_encode = box.internal.tuple.encode local tuple_bless = box.internal.tuple.bless local is_tuple = box.tuple.is assert(tuple_encode ~= nil and tuple_bless ~= nil and is_tuple ~= nil) +local cord_ibuf_take = buffer.internal.cord_ibuf_take +local cord_ibuf_put = buffer.internal.cord_ibuf_put local INT64_MIN = tonumber64('-9223372036854775808') local INT64_MAX = tonumber64('9223372036854775807') @@ -1785,9 +1787,12 @@ base_index_mt.__len = base_index_mt.len -- min and max base_index_mt.min_ffi = function(index, key) check_index_arg(index, 'min') - local pkey, pkey_end = tuple_encode(key) - if builtin.box_index_min(index.space_id, index.id, - pkey, pkey_end, ptuple) ~= 0 then + local ibuf = cord_ibuf_take() + local pkey, pkey_end = tuple_encode(ibuf, key) + local nok = builtin.box_index_min(index.space_id, index.id, pkey, pkey_end, + ptuple) ~= 0 + cord_ibuf_put(ibuf) + if nok then box.error() -- error elseif ptuple[0] ~= nil then return tuple_bless(ptuple[0]) @@ -1802,9 +1807,12 @@ base_index_mt.min_luac = function(index, key) end base_index_mt.max_ffi = function(index, key) check_index_arg(index, 'max') - local pkey, pkey_end = tuple_encode(key) - if builtin.box_index_max(index.space_id, index.id, - pkey, pkey_end, ptuple) ~= 0 then + local ibuf = cord_ibuf_take() + local pkey, pkey_end = tuple_encode(ibuf, key) + local nok = builtin.box_index_max(index.space_id, index.id, pkey, pkey_end, + ptuple) ~= 0 + cord_ibuf_put(ibuf) + if nok then box.error() -- error elseif ptuple[0] ~= nil then return tuple_bless(ptuple[0]) @@ -1837,10 +1845,12 @@ end -- iteration base_index_mt.pairs_ffi = function(index, key, opts) check_index_arg(index, 'pairs') - local pkey, pkey_end = tuple_encode(key) + local ibuf = cord_ibuf_take() + local pkey, pkey_end = tuple_encode(ibuf, key) local itype = check_iterator_type(opts, pkey + 1 >= pkey_end); local keybuf = ffi.string(pkey, pkey_end - pkey) + cord_ibuf_put(ibuf) local pkeybuf = ffi.cast('const char *', keybuf) local cdata = builtin.box_index_iterator(index.space_id, index.id, itype, pkeybuf, pkeybuf + #keybuf); @@ -1864,10 +1874,12 @@ end -- index subtree size base_index_mt.count_ffi = function(index, key, opts) check_index_arg(index, 'count') - local pkey, pkey_end = tuple_encode(key) + local ibuf = cord_ibuf_take() + local pkey, pkey_end = tuple_encode(ibuf, key) local itype = check_iterator_type(opts, pkey + 1 >= pkey_end); local count = builtin.box_index_count(index.space_id, index.id, itype, pkey, pkey_end); + cord_ibuf_put(ibuf) if count == -1 then box.error() end @@ -1882,9 +1894,12 @@ end base_index_mt.get_ffi = function(index, key) check_index_arg(index, 'get') - local key, key_end = tuple_encode(key) - if builtin.box_index_get(index.space_id, index.id, - key, key_end, ptuple) ~= 0 then + local ibuf = cord_ibuf_take() + local key, key_end = tuple_encode(ibuf, key) + local nok = builtin.box_index_get(index.space_id, index.id, key, key_end, + ptuple) ~= 0 + cord_ibuf_put(ibuf) + if nok then return box.error() -- error elseif ptuple[0] ~= nil then return tuple_bless(ptuple[0]) @@ -1915,13 +1930,15 @@ end base_index_mt.select_ffi = function(index, key, opts) check_index_arg(index, 'select') - local key, key_end = tuple_encode(key) + local ibuf = cord_ibuf_take() + local key, key_end = tuple_encode(ibuf, key) local iterator, offset, limit = check_select_opts(opts, key + 1 >= key_end) local port = ffi.cast('struct port *', port_c) - - if builtin.box_select(index.space_id, index.id, - iterator, offset, limit, key, key_end, port) ~= 0 then + local nok = builtin.box_select(index.space_id, index.id, iterator, offset, + limit, key, key_end, port) ~= 0 + cord_ibuf_put(ibuf) + if nok then return box.error() end diff --git a/src/box/lua/tuple.c b/src/box/lua/tuple.c index ffd01d899..f7198a025 100644 --- a/src/box/lua/tuple.c +++ b/src/box/lua/tuple.c @@ -36,6 +36,7 @@ #include "diag.h" /* diag_set() */ #include #include +#include "cord_buf.h" #include #include "box/tuple.h" @@ -280,16 +281,18 @@ luaT_tuple_encode(struct lua_State *L, int idx, size_t *tuple_len_ptr) box_tuple_t * luaT_tuple_new(struct lua_State *L, int idx, box_tuple_format_t *format) { - struct ibuf *ibuf = tarantool_lua_ibuf; - ibuf_reset(ibuf); + struct ibuf *ibuf = cord_ibuf_take(); size_t tuple_len; + box_tuple_t *tuple; char *tuple_data = luaT_tuple_encode_on_lua_ibuf(L, idx, &tuple_len, ibuf); - if (tuple_data == NULL) - return NULL; - box_tuple_t *tuple = box_tuple_new(format, tuple_data, - tuple_data + tuple_len); - ibuf_reinit(ibuf); + if (tuple_data == NULL) { + tuple = NULL; + } else { + tuple = box_tuple_new(format, tuple_data, + tuple_data + tuple_len); + } + cord_ibuf_drop(ibuf); return tuple; } @@ -308,12 +311,11 @@ lbox_tuple_new(lua_State *L) */ box_tuple_format_t *fmt = box_tuple_format_default(); if (argc != 1 || (!lua_istable(L, 1) && !luaT_istuple(L, 1))) { - struct ibuf *buf = tarantool_lua_ibuf; - ibuf_reset(buf); + struct ibuf *buf = cord_ibuf_take(); luaT_tuple_encode_values(L, buf); /* may raise */ struct tuple *tuple = box_tuple_new(fmt, buf->buf, buf->buf + ibuf_used(buf)); - ibuf_reinit(buf); + cord_ibuf_drop(buf); if (tuple == NULL) return luaT_error(L); luaT_pushtuple(L, tuple); @@ -573,8 +575,7 @@ lbox_tuple_transform(struct lua_State *L) return 1; } - struct ibuf *buf = tarantool_lua_ibuf; - ibuf_reset(buf); + struct ibuf *buf = cord_ibuf_take(); struct mpstream stream; mpstream_init(&stream, buf, ibuf_reserve_cb, ibuf_alloc_cb, luamp_error, L); @@ -619,12 +620,11 @@ lbox_tuple_transform(struct lua_State *L) new_tuple = tuple_new(box_tuple_format_default(), new_data, new_data + new_size); region_truncate(region, used); - + cord_ibuf_put(buf); if (new_tuple == NULL) luaT_error(L); luaT_pushtuple(L, new_tuple); - ibuf_reset(buf); return 1; } diff --git a/src/box/lua/tuple.lua b/src/box/lua/tuple.lua index 813d66ff0..fa76f4f7f 100644 --- a/src/box/lua/tuple.lua +++ b/src/box/lua/tuple.lua @@ -5,6 +5,8 @@ local msgpackffi = require('msgpackffi') local fun = require('fun') local buffer = require('buffer') local internal = require('box.internal') +local cord_ibuf_take = buffer.internal.cord_ibuf_take +local cord_ibuf_put = buffer.internal.cord_ibuf_put ffi.cdef[[ /** \cond public */ @@ -71,9 +73,7 @@ local encode_fix = msgpackffi.internal.encode_fix local encode_array = msgpackffi.internal.encode_array local encode_r = msgpackffi.internal.encode_r -local tuple_encode = function(obj) - local tmpbuf = buffer.IBUF_SHARED - tmpbuf:reset() +local tuple_encode = function(tmpbuf, obj) if obj == nil then encode_fix(tmpbuf, 0x90, 0) -- empty array elseif is_tuple(obj) then @@ -229,8 +229,10 @@ local function tuple_update(tuple, expr) if type(expr) ~= 'table' then error("Usage: tuple:update({ { op, field, arg}+ })") end - local pexpr, pexpr_end = tuple_encode(expr) + local ibuf = cord_ibuf_take() + local pexpr, pexpr_end = tuple_encode(ibuf, expr) local tuple = builtin.box_tuple_update(tuple, pexpr, pexpr_end) + cord_ibuf_put(ibuf) if tuple == nil then return box.error() end @@ -242,8 +244,10 @@ local function tuple_upsert(tuple, expr) if type(expr) ~= 'table' then error("Usage: tuple:upsert({ { op, field, arg}+ })") end - local pexpr, pexpr_end = tuple_encode(expr) + local ibuf = cord_ibuf_take() + local pexpr, pexpr_end = tuple_encode(ibuf, expr) local tuple = builtin.box_tuple_upsert(tuple, pexpr, pexpr_end) + cord_ibuf_put(ibuf) if tuple == nil then return box.error() end diff --git a/src/exports.h b/src/exports.h index eb72bc928..ddbe57230 100644 --- a/src/exports.h +++ b/src/exports.h @@ -113,6 +113,9 @@ EXPORT(coio_getaddrinfo) EXPORT(coio_wait) EXPORT(console_get_output_format) EXPORT(console_set_output_format) +EXPORT(cord_ibuf_drop) +EXPORT(cord_ibuf_put) +EXPORT(cord_ibuf_take) EXPORT(cord_slab_cache) EXPORT(crc32_calc) EXPORT(crypto_EVP_MD_CTX_free) @@ -493,7 +496,6 @@ EXPORT(swim_set_codec) EXPORT(swim_set_payload) EXPORT(swim_size) EXPORT(tarantool_exit) -EXPORT(tarantool_lua_ibuf) EXPORT(tarantool_lua_slab_cache) EXPORT(tarantool_uptime) EXPORT(title_get) diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt index 30cf0dd15..2cd4d0b4f 100644 --- a/src/lib/core/CMakeLists.txt +++ b/src/lib/core/CMakeLists.txt @@ -29,6 +29,7 @@ set(core_sources port.c decimal.c mp_decimal.c + cord_buf.c ) if (TARGET_OS_NETBSD) diff --git a/src/lib/core/cord_buf.c b/src/lib/core/cord_buf.c new file mode 100644 index 000000000..cac508c3d --- /dev/null +++ b/src/lib/core/cord_buf.c @@ -0,0 +1,47 @@ +/* + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file. + */ +#include "cord_buf.h" +#include "fiber.h" + +#include "small/ibuf.h" + +enum { + /* No any reason why that value. Historical constant. */ + CORD_IBUF_START_CAPACITY = 16384, +}; + +static struct ibuf *cord_buf_global = NULL; + +struct ibuf * +cord_ibuf_take(void) +{ + assert(cord_is_main()); + struct ibuf *buf = cord_buf_global; + if (buf != NULL) { + ibuf_reset(buf); + return buf; + } + buf = malloc(sizeof(*buf)); + if (buf == NULL) + panic("Couldn't allocate thread buffer"); + ibuf_create(buf, &cord()->slabc, CORD_IBUF_START_CAPACITY); + cord_buf_global = buf; + return buf; +} + +void +cord_ibuf_put(struct ibuf *ibuf) +{ + (void)ibuf; + assert(ibuf == cord_buf_global); +} + +void +cord_ibuf_drop(struct ibuf *ibuf) +{ + ibuf_reinit(ibuf); + assert(ibuf == cord_buf_global); +} diff --git a/src/lib/core/cord_buf.h b/src/lib/core/cord_buf.h new file mode 100644 index 000000000..59f429c8f --- /dev/null +++ b/src/lib/core/cord_buf.h @@ -0,0 +1,45 @@ +/* + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file. + */ +#pragma once + +#if defined(__cplusplus) +extern "C" { +#endif /* defined(__cplusplus) */ + +struct ibuf; + +/** + * Take the global ibuf, or allocate a new one if the stash is empty. + */ +struct ibuf * +cord_ibuf_take(void); + +/** + * Put the global ibuf back. + */ +void +cord_ibuf_put(struct ibuf *ibuf); + +/** + * Put the global ibuf back and free its memory. So only the buffer object + * itself is saved to the stash. Main reason why it is a dedicated function is + * because it is often needed from Lua, and allows not to call :recycle() there, + * which would be an additional FFI call before cord_ibuf_put(). + * + * XXX: recycle of the global buffer is a workaround for the ibuf being used in + * some places working with Lua API, where it wasn't wanted to "reuse" it + * anyhow. Instead, the global buffer is used to protect from the buffer leak in + * case it would be created locally, and then a Lua error would be raised. When + * the buffer is global, it is not a problem, because it is reused/recycled + * later. But it hurts the places, where re-usage could be good. Probably it is + * worth to separate take/put() from new/drop() API. Or delete drop() entirely. + */ +void +cord_ibuf_drop(struct ibuf *ibuf); + +#if defined(__cplusplus) +} +#endif /* defined(__cplusplus) */ diff --git a/src/lua/buffer.lua b/src/lua/buffer.lua index 00846bb20..d5dbedb0a 100644 --- a/src/lua/buffer.lua +++ b/src/lua/buffer.lua @@ -7,7 +7,15 @@ ffi.cdef[[ struct slab_cache; struct slab_cache * tarantool_lua_slab_cache(); -extern struct ibuf *tarantool_lua_ibuf; + +struct ibuf * +cord_ibuf_take(void); + +void +cord_ibuf_put(struct ibuf *ibuf); + +void +cord_ibuf_drop(struct ibuf *ibuf); struct ibuf { @@ -244,9 +252,48 @@ end -- local reg_array = ffi.new('union c_register[?]', 2) +-- +-- Cord buffer is useful for the places, where +-- +-- * Want to reuse the already allocated memory which might be stored in the +-- cord buf. Although sometimes the buffer is recycled, so should not rely on +-- being able to reuse it always. When reused, the win is the biggest - +-- becomes about x20 times faster than a new buffer creation (~5ns vs ~100ns); +-- +-- * Want to avoid allocation of a new ibuf because it produces a new GC object +-- which is additional load for Lua GC. Although according to benches it is +-- not super expensive; +-- +-- * Almost always can put the buffer back manually. Not rely on it being +-- recycled automatically. It is recycled, but still should not rely on that; +-- +-- It is important to wrap the C functions, not expose them directly. Because +-- JIT works a bit better when C functions are called as 'ffi.C.func()' than +-- 'func()' with func being cached. The only pros is to cache 'ffi.C' itself. +-- It is quite strange though how having them wrapped into a Lua function is +-- faster than cached directly as C functions. +-- +local function cord_ibuf_take() + return builtin.cord_ibuf_take() +end + +local function cord_ibuf_put(buf) + return builtin.cord_ibuf_put(buf) +end + +local function cord_ibuf_drop(buf) + return builtin.cord_ibuf_drop(buf) +end + +local internal = { + cord_ibuf_take = cord_ibuf_take, + cord_ibuf_put = cord_ibuf_put, + cord_ibuf_drop = cord_ibuf_drop, +} + return { + internal = internal, ibuf = ibuf_new; - IBUF_SHARED = ffi.C.tarantool_lua_ibuf; READAHEAD = READAHEAD; static_alloc = static_alloc, -- Keep reference. diff --git a/src/lua/iconv.lua b/src/lua/iconv.lua index e68509dec..732b80514 100644 --- a/src/lua/iconv.lua +++ b/src/lua/iconv.lua @@ -1,6 +1,8 @@ local ffi = require('ffi') local errno = require('errno') local buffer = require('buffer') +local cord_ibuf_take = buffer.internal.cord_ibuf_take +local cord_ibuf_put = buffer.internal.cord_ibuf_put ffi.cdef[[ typedef struct iconv *iconv_t; @@ -33,10 +35,9 @@ local function iconv_convert(iconv, data) -- prepare at lease BUF_SIZE and at most data_len bytes in shared buffer local output_len = data_len >= BUF_SIZE and data_len or BUF_SIZE - local buf = buffer.IBUF_SHARED; + local buf = cord_ibuf_take(); local buf_ptr = char_ptr_arr_t() local buf_left = size_t_arr_t() - buf:reset() while data_left[0] > 0 do buf_ptr[0] = buf:reserve(output_len) @@ -46,6 +47,7 @@ local function iconv_convert(iconv, data) if res == ffi.cast('size_t', -1) then local err = errno() if err ~= E2BIG then + cord_ibuf_put(buf) ffi.C.tnt_iconv(iconv, nil, nil, nil, nil) if err == EINVAL then error('Invalid multibyte sequence') @@ -62,7 +64,7 @@ local function iconv_convert(iconv, data) -- iconv function sets cd's conversion state to the initial state ffi.C.tnt_iconv(iconv, nil, nil, nil, nil) local result = ffi.string(buf.rpos, buf:size()) - buf:reset() + cord_ibuf_put(buf) return result end diff --git a/src/lua/init.c b/src/lua/init.c index 767abdfc5..89d7f8f73 100644 --- a/src/lua/init.c +++ b/src/lua/init.c @@ -73,8 +73,6 @@ * The single Lua state of the transaction processor (tx) thread. */ struct lua_State *tarantool_L; -static struct ibuf tarantool_lua_ibuf_body; -struct ibuf *tarantool_lua_ibuf = &tarantool_lua_ibuf_body; /** * The fiber running the startup Lua script */ @@ -453,7 +451,6 @@ tarantool_lua_init(const char *tarantool_bin, int argc, char **argv) if (L == NULL) { panic("failed to initialize Lua"); } - ibuf_create(tarantool_lua_ibuf, tarantool_lua_slab_cache(), 16000); luaL_openlibs(L); tarantool_lua_setpaths(L); diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c index 24b0d2ccd..1e74a6a3c 100644 --- a/src/lua/msgpack.c +++ b/src/lua/msgpack.c @@ -46,6 +46,7 @@ #include "lib/uuid/mp_uuid.h" /* mp_decode_uuid() */ #include "lib/core/mp_extension_types.h" +#include "cord_buf.h" #include void @@ -362,8 +363,7 @@ lua_msgpack_encode(lua_State *L) "must be of type 'struct ibuf'"); } } else { - buf = tarantool_lua_ibuf; - ibuf_reset(buf); + buf = cord_ibuf_take(); } size_t used = ibuf_used(buf); @@ -380,7 +380,7 @@ lua_msgpack_encode(lua_State *L) lua_pushinteger(L, ibuf_used(buf) - used); } else { lua_pushlstring(L, buf->buf, ibuf_used(buf)); - ibuf_reinit(buf); + cord_ibuf_drop(buf); } return 1; } diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua index 2bc374742..6c245d7aa 100644 --- a/src/lua/msgpackffi.lua +++ b/src/lua/msgpackffi.lua @@ -10,6 +10,8 @@ local uint16_ptr_t = ffi.typeof('uint16_t *') local uint32_ptr_t = ffi.typeof('uint32_t *') local uint64_ptr_t = ffi.typeof('uint64_t *') local char_ptr_t = ffi.typeof('char *') +local cord_ibuf_take = buffer.internal.cord_ibuf_take +local cord_ibuf_drop = buffer.internal.cord_ibuf_drop ffi.cdef([[ char * @@ -294,11 +296,10 @@ local function encode_r(buf, obj, level) end local function encode(obj) - local tmpbuf = buffer.IBUF_SHARED - tmpbuf:reset() + local tmpbuf = cord_ibuf_take() encode_r(tmpbuf, obj, 0) local r = ffi.string(tmpbuf.rpos, tmpbuf:size()) - tmpbuf:recycle() + cord_ibuf_drop(tmpbuf) return r end diff --git a/src/lua/socket.lua b/src/lua/socket.lua index 78d91f010..7c24b5655 100644 --- a/src/lua/socket.lua +++ b/src/lua/socket.lua @@ -13,6 +13,8 @@ local buffer = require('buffer') local reg1 = buffer.reg1 local reg2 = buffer.reg2 local static_alloc = buffer.static_alloc +local cord_ibuf_take = buffer.internal.cord_ibuf_take +local cord_ibuf_drop = buffer.internal.cord_ibuf_drop local format = string.format @@ -296,19 +298,15 @@ local function socket_sysread(self, arg1, arg2) error('socket:sysread(): size can not be negative') end - local buf = buffer.IBUF_SHARED - buf:reset() + local buf = cord_ibuf_take() local p = buf:alloc(size) local res = sysread(self, p, size) if res then - local str = ffi.string(p, res) - buf:recycle() - return str - else - buf:recycle() - return res + res = ffi.string(p, res) end + cord_ibuf_drop(buf) + return res end local function socket_nonblock(self, nb) diff --git a/src/lua/swim.lua b/src/lua/swim.lua index 1da55337a..42b0d15ef 100644 --- a/src/lua/swim.lua +++ b/src/lua/swim.lua @@ -6,6 +6,8 @@ local crypto = require('crypto') local fiber = require('fiber') local internal = require('swim') local schedule_task = fiber._internal.schedule_task +local cord_ibuf_take = buffer.internal.cord_ibuf_take +local cord_ibuf_put = buffer.internal.cord_ibuf_put ffi.cdef[[ struct swim; @@ -655,14 +657,17 @@ end local function swim_set_payload(s, payload) local func_name = 'swim:set_payload' local ptr = swim_check_instance(s, func_name) - local payload_size = 0 - if payload ~= nil then - local buf = buffer.IBUF_SHARED - buf:reset() - payload_size = msgpack.encode(payload, buf) + local rc + if payload == nil then + rc = capi.swim_set_payload(ptr, nil, 0) + else + local buf = cord_ibuf_take() + local payload_size = msgpack.encode(payload, buf) payload = buf.rpos + rc = capi.swim_set_payload(ptr, payload, payload_size) + cord_ibuf_put(buf) end - if capi.swim_set_payload(ptr, payload, payload_size) ~= 0 then + if rc ~= 0 then return nil, box.error.last() end return true diff --git a/src/lua/utf8.c b/src/lua/utf8.c index 9c678cad4..bf9bb98f4 100644 --- a/src/lua/utf8.c +++ b/src/lua/utf8.c @@ -33,12 +33,11 @@ #include "coll/coll.h" #include "lua/utils.h" #include "lua/utf8.h" +#include "cord_buf.h" #include "diag.h" #include "small/ibuf.h" #include "tt_static.h" -extern struct ibuf *tarantool_lua_ibuf; - /** Collations for cmp/casecmp functions. */ static struct coll *unicode_coll = NULL; static struct coll *unicode_ci_coll = NULL; @@ -52,12 +51,13 @@ utf8_str_to_case(struct lua_State *L, const char *src, int src_bsize, int i = 0; int dst_bsize = src_bsize; (void) i; + struct ibuf *ibuf = cord_ibuf_take(); do { UErrorCode err = U_ZERO_ERROR; - ibuf_reset(tarantool_lua_ibuf); - char *dst = ibuf_alloc(tarantool_lua_ibuf, dst_bsize); + char *dst = ibuf_alloc(ibuf, dst_bsize); if (dst == NULL) { diag_set(OutOfMemory, dst_bsize, "ibuf_alloc", "dst"); + cord_ibuf_put(ibuf); return luaT_error(L); } int real_bsize; @@ -73,11 +73,13 @@ utf8_str_to_case(struct lua_State *L, const char *src, int src_bsize, if (err == U_ZERO_ERROR || err == U_STRING_NOT_TERMINATED_WARNING) { lua_pushlstring(L, dst, real_bsize); + cord_ibuf_put(ibuf); return 1; } else if (err == U_BUFFER_OVERFLOW_ERROR) { assert(real_bsize > dst_bsize); dst_bsize = real_bsize; } else { + cord_ibuf_put(ibuf); lua_pushnil(L); lua_pushstring(L, tt_sprintf("error during ICU case "\ "transform: %s", @@ -249,9 +251,10 @@ utf8_char(struct lua_State *L) return 1; } /* Slow way - use dynamic buffer. */ - ibuf_reset(tarantool_lua_ibuf); - char *str = ibuf_alloc(tarantool_lua_ibuf, top * U8_MAX_LENGTH); + struct ibuf *ibuf = cord_ibuf_take(); + char *str = ibuf_alloc(ibuf, top * U8_MAX_LENGTH); if (str == NULL) { + cord_ibuf_put(ibuf); diag_set(OutOfMemory, top * U8_MAX_LENGTH, "ibuf_alloc", "str"); return luaT_error(L); @@ -261,6 +264,7 @@ utf8_char(struct lua_State *L) U8_APPEND_UNSAFE(str, len, c); } lua_pushlstring(L, str, len); + cord_ibuf_put(ibuf); return 1; } diff --git a/src/lua/utils.h b/src/lua/utils.h index 37531676d..4a164868b 100644 --- a/src/lua/utils.h +++ b/src/lua/utils.h @@ -72,7 +72,6 @@ struct tt_uuid; * snprintf(m_errmsg, sizeof(m_errmsg), "%s", msg ? msg : ""); */ extern struct lua_State *tarantool_L; -extern struct ibuf *tarantool_lua_ibuf; struct tt_uuid * luaL_pushuuid(struct lua_State *L); diff --git a/static-build/test/static-build/exports.test.lua b/static-build/test/static-build/exports.test.lua index 9b9eaa471..d5dbe7de9 100755 --- a/static-build/test/static-build/exports.test.lua +++ b/static-build/test/static-build/exports.test.lua @@ -26,6 +26,9 @@ local check_symbols = { 'guava', 'base64_decode', 'base64_encode', + 'cord_ibuf_drop', + 'cord_ibuf_put', + 'cord_ibuf_take', 'SHA1internal', 'random_bytes', 'fiber_time', @@ -42,7 +45,6 @@ local check_symbols = { 'exception_get_int', 'exception_get_string', - 'tarantool_lua_ibuf', 'uuid_nil', 'tt_uuid_create', 'tt_uuid_str', diff --git a/test/unit/luaT_tuple_new.c b/test/unit/luaT_tuple_new.c index 965b2e6e0..1d7c9072a 100644 --- a/test/unit/luaT_tuple_new.c +++ b/test/unit/luaT_tuple_new.c @@ -25,8 +25,6 @@ * box/tuple.test.lua. */ -extern struct ibuf *tarantool_lua_ibuf; - uint32_t min_u32(uint32_t a, uint32_t b) { @@ -184,8 +182,6 @@ main() memory_init(); fiber_init(fiber_c_invoke); - ibuf_create(tarantool_lua_ibuf, &cord()->slabc, 16000); - struct lua_State *L = luaL_newstate(); luaL_openlibs(L); -- 2.24.3 (Apple Git-128)