Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
To: tarantool-patches@freelists.org
Cc: vdavydov.dev@gmail.com
Subject: [PATCH 0/8] netbox: introduce fiber-async API
Date: Mon, 16 Apr 2018 21:39:10 +0300	[thread overview]
Message-ID: <cover.1523903144.git.v.shpilevoy@tarantool.org> (raw)

Branch: http://github.com/tarantool/tarantool/tree/gh-3107-async-netbox
Issue: https://github.com/tarantool/tarantool/issues/3107

The patchset introduces a fiber-async API for netbox, but before
a lot of preparation work is done.

At first, the box.error module is fixed by repairing of
box.error.raise() and introducing box.error.new(). Box.error.new()
is needed for an async netbox future object to be able to save an
error object until a user requests a result. The second necessity
in box.error.new() is that with no error object a netbox can not
satisfy an error returning policy: in a case of error return nil
and error object.

At second, the interactive console is fixed to be able to detect a
spurious wakeup. It was needed for a first version of async
netbox, but then it was removed from the final version, and the
patch remains.

At third, a real netbox codecs are introduced. Netbox async future
object must be able to decode a response out of call context, when
the future object is already returned to a user. The future object
must be able to decode a response by only raw data and method
name. Moreover, it allows to remove double decoding of all
requests for https://github.com/tarantool/tarantool/issues/3333
(it is not implemented here - another patchset is needed).

At fourth, the async API is introduced.

Now any netbox call blocks a caller-fiber until a result is read
from a socket, or time is out. To use it asynchronously it is
necessary to create a fiber per request. Sometimes it is
unwanted - for example if RPS is very high (for example, about
100k), and latency is about 1 second. Or when it is neccessary
to send multiple requests in paralles and then collect responses
(map-reduce).

The patchset introduces a new option for all netbox requests:
is_async. With this option any called netbox method returns
immediately (but still yields for a moment) a 'future' object.

By a future object a user can check if the request is finalized,
get a result or error, wait for a timeout, discard a response.

Example of is_async usage:
future = conn:call(func, {params}, {..., is_async = true})
-- Do some work ...
if not future.is_ready() then
    result, err = future:wait_result(timeout)
end
-- Or:
result, error = future:result()

A future:result() and :wait_result() returns either an error or
a response in the same format, as the sync versions of the called
methods.

Vladislav Shpilevoy (8):
  lua: fix box.error.raise
  lua: allow to create and error object with no throw
  console: fix a bug in interactive readline usage
  netbox: extend codec with 'decode' methods
  test: fix unstable test
  netbox: introduce fiber-async API
  netbox: remove schema_version from requests
  netbox: implement perform_request via async version

 src/box/lua/console.c     |   5 +-
 src/box/lua/error.cc      |  93 +++++---
 src/box/lua/net_box.c     | 105 ++++-----
 src/box/lua/net_box.lua   | 349 ++++++++++++++++++++----------
 test/box/errinj.result    |   3 +
 test/box/errinj.test.lua  |   1 +
 test/box/misc.result      |  64 ++++++
 test/box/misc.test.lua    |  28 +++
 test/box/net.box.result   | 537 +++++++++++++++++++++++++++++++++++++++++++++-
 test/box/net.box.test.lua | 190 +++++++++++++++-
 test/box/sql.result       |   2 +
 11 files changed, 1169 insertions(+), 208 deletions(-)

-- 
2.15.1 (Apple Git-101)

             reply	other threads:[~2018-04-16 18:39 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-16 18:39 Vladislav Shpilevoy [this message]
2018-04-16 18:39 ` [PATCH 1/8] lua: fix box.error.raise Vladislav Shpilevoy
2018-04-23 16:19   ` Vladimir Davydov
2018-05-08 15:36   ` [tarantool-patches] " Konstantin Osipov
2018-05-08 17:24     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-16 18:39 ` [PATCH 2/8] lua: allow to create and error object with no throw Vladislav Shpilevoy
2018-04-23 16:20   ` Vladimir Davydov
2018-05-08 15:37   ` [tarantool-patches] " Konstantin Osipov
2018-04-16 18:39 ` [PATCH 3/8] console: fix a bug in interactive readline usage Vladislav Shpilevoy
2018-04-23 16:20   ` Vladimir Davydov
2018-05-08 15:37   ` [tarantool-patches] " Konstantin Osipov
2018-04-16 18:39 ` [PATCH 4/8] netbox: extend codec with 'decode' methods Vladislav Shpilevoy
2018-04-23 16:42   ` Vladimir Davydov
2018-04-23 18:59     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-24 13:16       ` Vladimir Davydov
2018-05-08 15:49   ` [tarantool-patches] " Konstantin Osipov
2018-05-08 17:24     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-16 18:39 ` [PATCH 5/8] test: fix unstable test Vladislav Shpilevoy
2018-04-22  5:32   ` [tarantool-patches] " Kirill Yukhin
2018-05-08 15:50   ` Konstantin Osipov
2018-04-16 18:39 ` [PATCH 6/8] netbox: introduce fiber-async API Vladislav Shpilevoy
2018-04-23 12:31   ` [tarantool-patches] " Alexander Turenko
2018-04-23 18:59     ` Vladislav Shpilevoy
2018-04-23 16:44   ` Vladimir Davydov
2018-04-23 18:59     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-24 13:05       ` Vladimir Davydov
2018-04-16 18:39 ` [PATCH 7/8] netbox: remove schema_version from requests Vladislav Shpilevoy
2018-05-08 16:06   ` [tarantool-patches] " Konstantin Osipov
2018-05-08 17:24     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-16 18:39 ` [PATCH 8/8] netbox: implement perform_request via async version Vladislav Shpilevoy
2018-04-23 16:47   ` Vladimir Davydov
2018-04-23 19:00     ` [tarantool-patches] " Vladislav Shpilevoy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1523903144.git.v.shpilevoy@tarantool.org \
    --to=v.shpilevoy@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --cc=vdavydov.dev@gmail.com \
    --subject='Re: [PATCH 0/8] netbox: introduce fiber-async API' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox