From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 787CF6EC58; Tue, 3 Aug 2021 00:49:54 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 787CF6EC58 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1627940994; bh=45MtrbQhCpKvqBVRiP0A2jH58esdiuM8e5M698J/kzU=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Mw/xrEzEIqVMrXCDP8XUjXFa/WGGlD2mQr38AC9u6UTBj0XiADMtCR49ZRx2AAwFL 0v3VHCmS0wfh7IBv9HRBU0pAjXW3cqJxVTLKzXAIlsuqaHq2Uop7qIQERDfMdFt5/S koFMVCfCOVXfdOHZQXeNoCcdNQiu64U1uzCdU2eg= Received: from smtpng3.i.mail.ru (smtpng3.i.mail.ru [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 57C686EC58 for ; Tue, 3 Aug 2021 00:49:53 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 57C686EC58 Received: by smtpng3.m.smailru.net with esmtpa (envelope-from ) id 1mAfom-0004qI-65; Tue, 03 Aug 2021 00:49:52 +0300 To: Vladimir Davydov , tarantool-patches@dev.tarantool.org References: <5bc68e715031c04ebb022c70cf4ff27c0939598c.1627024646.git.vdavydov@tarantool.org> Message-ID: <9dbecb42-b2c8-ff46-ef1c-c8c14f19357f@tarantool.org> Date: Mon, 2 Aug 2021 23:49:51 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <5bc68e715031c04ebb022c70cf4ff27c0939598c.1627024646.git.vdavydov@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD941C43E597735A9C3FDAB68B812060C77E97CF8617D978122182A05F5380850402EBB4C6F5601E842E16EC5B75E9EDD521B214ECD4262F483B2BF51B1F39E5B7D X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7A33E1178EA603666EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006374C29A2D8B98EBF198638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D80E6FC581712D3F68051101E5AA8E9479117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8B2EE5AD8F952D28FBA471835C12D1D977C4224003CC8364762BB6847A3DEAEFB0F43C7A68FF6260569E8FC8737B5C2249EC8D19AE6D49635B68655334FD4449CB9ECD01F8117BC8BEAAAE862A0553A39223F8577A6DFFEA7C565C1E6824D8037B43847C11F186F3C59DAA53EE0834AAEE X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B5056030AD0FFA255090D99DC214C1B0E5FC X-C1DE0DAB: 0D63561A33F958A51BA1F42221E8711749030DB85A9FD033260ECC9E756EE21DD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA751B940EDA0DFB0535410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34D2AC226BFD455724D320F25DC2C38468549DBA6F2E7B367EAAC9DBD0C160191AEE7DCDD9FE7867631D7E09C32AA3244CB2A71DC94A0E475D43133701298FF63505AB220A9D022EBC729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj9N286KAyvN5v4hVbKN35Hw== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5D999A8C3D90AED2A42AE5987A7C78AFDD3841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 13/20] net.box: rewrite send_and_recv_{iproto, console} in C X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thanks for working on this! See 3 comments below. > diff --git a/src/box/lua/net_box.c b/src/box/lua/net_box.c > index e88db6323afa..12d82738a050 100644 > --- a/src/box/lua/net_box.c > +++ b/src/box/lua/net_box.c > @@ -465,62 +465,45 @@ netbox_decode_greeting(lua_State *L) > * interaction. > */ > static int > -netbox_communicate(lua_State *L) > +netbox_communicate(int fd, struct ibuf *send_buf, struct ibuf *recv_buf, > + size_t limit, const void *boundary, size_t boundary_len, > + double timeout, size_t *limit_or_boundary_pos) 1. IMO, the name looks really bulky. I would just call it `size` or `end_pos`. Up to you. > { > - uint32_t fd = lua_tonumber(L, 1); > const int NETBOX_READAHEAD = 16320; > - struct ibuf *send_buf = (struct ibuf *) lua_topointer(L, 2); > - struct ibuf *recv_buf = (struct ibuf *) lua_topointer(L, 3); > - > - /* limit or boundary */ > - size_t limit = SIZE_MAX; > - const void *boundary = NULL; > - size_t boundary_len; > - > - if (lua_type(L, 4) == LUA_TSTRING) > - boundary = lua_tolstring(L, 4, &boundary_len); > - else > - limit = lua_tonumber(L, 4); > - > - /* timeout */ > - ev_tstamp timeout = TIMEOUT_INFINITY; > - if (lua_type(L, 5) == LUA_TNUMBER) > - timeout = lua_tonumber(L, 5); > if (timeout < 0) { > - lua_pushinteger(L, ER_TIMEOUT); > - lua_pushstring(L, "Timeout exceeded"); > - return 2; > + diag_set(ClientError, ER_TIMEOUT); > + return -1; > } > int revents = COIO_READ; > while (true) { > /* reader serviced first */ > check_limit: > if (ibuf_used(recv_buf) >= limit) { > - lua_pushnil(L); > - lua_pushinteger(L, (lua_Integer)limit); > - return 2; > + *limit_or_boundary_pos = limit; > + return 0; > } > const char *p; > if (boundary != NULL && (p = memmem( > recv_buf->rpos, > ibuf_used(recv_buf), > boundary, boundary_len)) != NULL) { > - lua_pushnil(L); > - lua_pushinteger(L, (lua_Integer)( > - p - recv_buf->rpos)); > - return 2; > + *limit_or_boundary_pos = p - recv_buf->rpos; > + return 0; > } > > while (revents & COIO_READ) { > void *p = ibuf_reserve(recv_buf, NETBOX_READAHEAD); > - if (p == NULL) > - luaL_error(L, "out of memory"); > + if (p == NULL) { > + diag_set(OutOfMemory, NETBOX_READAHEAD, > + "ibuf", "recv_buf"); 2. For OutOfMemory errors as a rule we use the format diag_set(OutOfMemory, size, "name of the allocation function", "name of the variable") So here it would be diag_set(OutOfMemory, NETBOX_READAHEAD, "ibuf_reserve", "p"); I know it is violated in a lot of old code and I gave up trying to enforce it in new patches to exact that form. Up to you. The same in the other new OutOfMemory in the other patches. > diff --git a/src/box/lua/net_box.lua b/src/box/lua/net_box.lua > index d7394b088752..0ad6cac022f2 100644 > --- a/src/box/lua/net_box.lua > +++ b/src/box/lua/net_box.lua > @@ -582,46 +581,23 @@ local function create_transport(host, port, user, password, callback, > end > > -- IO (WORKER FIBER) -- > - local function send_and_recv(limit_or_boundary, timeout) > - return communicate(connection:fd(), send_buf, recv_buf, > - limit_or_boundary, timeout) > - end > - > local function send_and_recv_iproto(timeout) > - local data_len = recv_buf.wpos - recv_buf.rpos > - local required > - if data_len < 5 then > - required = 5 > - else > - -- PWN! insufficient input validation > - local bufpos = recv_buf.rpos > - local len, rpos = decode(bufpos) > - required = (rpos - bufpos) + len > - if data_len >= required then > - local body_end = rpos + len > - local hdr, body_rpos = decode(rpos) > - recv_buf.rpos = body_end > - return nil, hdr, body_rpos, body_end > - end > + local hdr, body_rpos, body_end = internal.send_and_recv_iproto( 3. Indexing 'internal' via '.' is not free. It is a lookup in a hash. You might want to save internal.send_and_recv_iproto into a global variable when the module loads first time and use the cached value. Non-cached version is a bit faster only for FFI, but here you are using Lua C - cache should be good. > + connection:fd(), send_buf, recv_buf, timeout) Another idea is to cache 'connection:fd()' result into a variable in the root of create_transport() function. And update it when the connetion is re-established. Although you probably move this all to C later as well, I didn't reach the last commits yet.