From: Vladislav Shpilevoy <v.shpilevoy@tarantool.org> To: tarantool-patches@freelists.org Cc: vdavydov.dev@gmail.com Subject: [PATCH 0/8] netbox: introduce fiber-async API Date: Mon, 16 Apr 2018 21:39:10 +0300 [thread overview] Message-ID: <cover.1523903144.git.v.shpilevoy@tarantool.org> (raw) Branch: http://github.com/tarantool/tarantool/tree/gh-3107-async-netbox Issue: https://github.com/tarantool/tarantool/issues/3107 The patchset introduces a fiber-async API for netbox, but before a lot of preparation work is done. At first, the box.error module is fixed by repairing of box.error.raise() and introducing box.error.new(). Box.error.new() is needed for an async netbox future object to be able to save an error object until a user requests a result. The second necessity in box.error.new() is that with no error object a netbox can not satisfy an error returning policy: in a case of error return nil and error object. At second, the interactive console is fixed to be able to detect a spurious wakeup. It was needed for a first version of async netbox, but then it was removed from the final version, and the patch remains. At third, a real netbox codecs are introduced. Netbox async future object must be able to decode a response out of call context, when the future object is already returned to a user. The future object must be able to decode a response by only raw data and method name. Moreover, it allows to remove double decoding of all requests for https://github.com/tarantool/tarantool/issues/3333 (it is not implemented here - another patchset is needed). At fourth, the async API is introduced. Now any netbox call blocks a caller-fiber until a result is read from a socket, or time is out. To use it asynchronously it is necessary to create a fiber per request. Sometimes it is unwanted - for example if RPS is very high (for example, about 100k), and latency is about 1 second. Or when it is neccessary to send multiple requests in paralles and then collect responses (map-reduce). The patchset introduces a new option for all netbox requests: is_async. With this option any called netbox method returns immediately (but still yields for a moment) a 'future' object. By a future object a user can check if the request is finalized, get a result or error, wait for a timeout, discard a response. Example of is_async usage: future = conn:call(func, {params}, {..., is_async = true}) -- Do some work ... if not future.is_ready() then result, err = future:wait_result(timeout) end -- Or: result, error = future:result() A future:result() and :wait_result() returns either an error or a response in the same format, as the sync versions of the called methods. Vladislav Shpilevoy (8): lua: fix box.error.raise lua: allow to create and error object with no throw console: fix a bug in interactive readline usage netbox: extend codec with 'decode' methods test: fix unstable test netbox: introduce fiber-async API netbox: remove schema_version from requests netbox: implement perform_request via async version src/box/lua/console.c | 5 +- src/box/lua/error.cc | 93 +++++--- src/box/lua/net_box.c | 105 ++++----- src/box/lua/net_box.lua | 349 ++++++++++++++++++++---------- test/box/errinj.result | 3 + test/box/errinj.test.lua | 1 + test/box/misc.result | 64 ++++++ test/box/misc.test.lua | 28 +++ test/box/net.box.result | 537 +++++++++++++++++++++++++++++++++++++++++++++- test/box/net.box.test.lua | 190 +++++++++++++++- test/box/sql.result | 2 + 11 files changed, 1169 insertions(+), 208 deletions(-) -- 2.15.1 (Apple Git-101)
next reply other threads:[~2018-04-16 18:39 UTC|newest] Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-04-16 18:39 Vladislav Shpilevoy [this message] 2018-04-16 18:39 ` [PATCH 1/8] lua: fix box.error.raise Vladislav Shpilevoy 2018-04-23 16:19 ` Vladimir Davydov 2018-05-08 15:36 ` [tarantool-patches] " Konstantin Osipov 2018-05-08 17:24 ` [tarantool-patches] " Vladislav Shpilevoy 2018-04-16 18:39 ` [PATCH 2/8] lua: allow to create and error object with no throw Vladislav Shpilevoy 2018-04-23 16:20 ` Vladimir Davydov 2018-05-08 15:37 ` [tarantool-patches] " Konstantin Osipov 2018-04-16 18:39 ` [PATCH 3/8] console: fix a bug in interactive readline usage Vladislav Shpilevoy 2018-04-23 16:20 ` Vladimir Davydov 2018-05-08 15:37 ` [tarantool-patches] " Konstantin Osipov 2018-04-16 18:39 ` [PATCH 4/8] netbox: extend codec with 'decode' methods Vladislav Shpilevoy 2018-04-23 16:42 ` Vladimir Davydov 2018-04-23 18:59 ` [tarantool-patches] " Vladislav Shpilevoy 2018-04-24 13:16 ` Vladimir Davydov 2018-05-08 15:49 ` [tarantool-patches] " Konstantin Osipov 2018-05-08 17:24 ` [tarantool-patches] " Vladislav Shpilevoy 2018-04-16 18:39 ` [PATCH 5/8] test: fix unstable test Vladislav Shpilevoy 2018-04-22 5:32 ` [tarantool-patches] " Kirill Yukhin 2018-05-08 15:50 ` Konstantin Osipov 2018-04-16 18:39 ` [PATCH 6/8] netbox: introduce fiber-async API Vladislav Shpilevoy 2018-04-23 12:31 ` [tarantool-patches] " Alexander Turenko 2018-04-23 18:59 ` Vladislav Shpilevoy 2018-04-23 16:44 ` Vladimir Davydov 2018-04-23 18:59 ` [tarantool-patches] " Vladislav Shpilevoy 2018-04-24 13:05 ` Vladimir Davydov 2018-04-16 18:39 ` [PATCH 7/8] netbox: remove schema_version from requests Vladislav Shpilevoy 2018-05-08 16:06 ` [tarantool-patches] " Konstantin Osipov 2018-05-08 17:24 ` [tarantool-patches] " Vladislav Shpilevoy 2018-04-16 18:39 ` [PATCH 8/8] netbox: implement perform_request via async version Vladislav Shpilevoy 2018-04-23 16:47 ` Vladimir Davydov 2018-04-23 19:00 ` [tarantool-patches] " Vladislav Shpilevoy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cover.1523903144.git.v.shpilevoy@tarantool.org \ --to=v.shpilevoy@tarantool.org \ --cc=tarantool-patches@freelists.org \ --cc=vdavydov.dev@gmail.com \ --subject='Re: [PATCH 0/8] netbox: introduce fiber-async API' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox