From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladislav Shpilevoy Subject: [PATCH 0/8] netbox: introduce fiber-async API Date: Mon, 16 Apr 2018 21:39:10 +0300 Message-Id: To: tarantool-patches@freelists.org Cc: vdavydov.dev@gmail.com List-ID: Branch: http://github.com/tarantool/tarantool/tree/gh-3107-async-netbox Issue: https://github.com/tarantool/tarantool/issues/3107 The patchset introduces a fiber-async API for netbox, but before a lot of preparation work is done. At first, the box.error module is fixed by repairing of box.error.raise() and introducing box.error.new(). Box.error.new() is needed for an async netbox future object to be able to save an error object until a user requests a result. The second necessity in box.error.new() is that with no error object a netbox can not satisfy an error returning policy: in a case of error return nil and error object. At second, the interactive console is fixed to be able to detect a spurious wakeup. It was needed for a first version of async netbox, but then it was removed from the final version, and the patch remains. At third, a real netbox codecs are introduced. Netbox async future object must be able to decode a response out of call context, when the future object is already returned to a user. The future object must be able to decode a response by only raw data and method name. Moreover, it allows to remove double decoding of all requests for https://github.com/tarantool/tarantool/issues/3333 (it is not implemented here - another patchset is needed). At fourth, the async API is introduced. Now any netbox call blocks a caller-fiber until a result is read from a socket, or time is out. To use it asynchronously it is necessary to create a fiber per request. Sometimes it is unwanted - for example if RPS is very high (for example, about 100k), and latency is about 1 second. Or when it is neccessary to send multiple requests in paralles and then collect responses (map-reduce). The patchset introduces a new option for all netbox requests: is_async. With this option any called netbox method returns immediately (but still yields for a moment) a 'future' object. By a future object a user can check if the request is finalized, get a result or error, wait for a timeout, discard a response. Example of is_async usage: future = conn:call(func, {params}, {..., is_async = true}) -- Do some work ... if not future.is_ready() then result, err = future:wait_result(timeout) end -- Or: result, error = future:result() A future:result() and :wait_result() returns either an error or a response in the same format, as the sync versions of the called methods. Vladislav Shpilevoy (8): lua: fix box.error.raise lua: allow to create and error object with no throw console: fix a bug in interactive readline usage netbox: extend codec with 'decode' methods test: fix unstable test netbox: introduce fiber-async API netbox: remove schema_version from requests netbox: implement perform_request via async version src/box/lua/console.c | 5 +- src/box/lua/error.cc | 93 +++++--- src/box/lua/net_box.c | 105 ++++----- src/box/lua/net_box.lua | 349 ++++++++++++++++++++---------- test/box/errinj.result | 3 + test/box/errinj.test.lua | 1 + test/box/misc.result | 64 ++++++ test/box/misc.test.lua | 28 +++ test/box/net.box.result | 537 +++++++++++++++++++++++++++++++++++++++++++++- test/box/net.box.test.lua | 190 +++++++++++++++- test/box/sql.result | 2 + 11 files changed, 1169 insertions(+), 208 deletions(-) -- 2.15.1 (Apple Git-101)