Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
To: tarantool-patches@freelists.org
Cc: vdavydov.dev@gmail.com
Subject: [PATCH v2 00/10] session: introduce box.session.push
Date: Fri, 20 Apr 2018 16:24:24 +0300	[thread overview]
Message-ID: <cover.1524228894.git.v.shpilevoy@tarantool.org> (raw)

Branch: http://github.com/tarantool/tarantool/tree/gh-2677-box-session-push
Issue: https://github.com/tarantool/tarantool/issues/2677

Sometimes it is needed to send some intermediate results from long poll request.
Or to create a watcher on a server side, which listens for some events and sends
notifications to a client. Or split a big response into multiple ones, as it is
described in Tarantool Wire Protocol RFC.

IPROTO_CHUNK protocol command solves all of the problems. IPROTO_CHUNK is a
non-final response, and a single request can produce multiple response chunks.
Box.session.push is API to send a chunk. It takes a Lua object and sends it to
a client using the way depending of a session type: console or binary.

Console session pushes are YAML documents tagged with using YAML %TAG feature -
push message is tagged with !push! tag. For example:
tarantool> box.session.push({a = 100, b = 200})
%TAG !push! tag:tarantool.io/push,2018
---       <--------- Here pushed message starts.
a: 100
b: 200
...
---       <--------- Here push() call result starts.
- true
...
It is not the same as future console.print() pushes - they are not linked with
chunks.

Binary push is an IProto response with IPROTO_CHUNK key in the REQUEST_TYPE
header field. In the rest it has the same structure as any another response.

The most of the patchset work is preparative.
First patches introduces YAML tags for Tarantool YAML codec library - they were
implemented already in libyaml, but was not used in Tarantool.

Next patches refactor session - virtual table and session meta depending on
session type are added. Session meta is used by IProto sessions to store latest
sync and connection, console session stores file descriptor of remote client,
other sessions store nothing. Session meta is accessed via session vtable.

IPROTO_CHUNK response must have the same sync as a request, and to avoid
specifying it in the box.session.push() API, that is ugly, the request sync is
stored in the local storage of the fiber that executes the request. Sync key
does not extend the storage size - instead it shares one of storage cells with
another fiber local variable, that is never used in IProto sessions.

Vladislav Shpilevoy (10):
  yaml: don't throw OOM on any error in yaml encoding
  yaml: introduce yaml.encode_tagged
  yaml: introduce yaml.decode_tag
  console: use Lua C API to do formatting for console
  session: move salt into iproto connection
  session: introduce session vtab and meta
  port: rename dump() into dump_msgpack()
  session: introduce text box.session.push
  session: enable box.session.push in local console
  session: introduce binary box.session.push

 src/box/authentication.cc                |   4 +-
 src/box/authentication.h                 |   3 +-
 src/box/box.cc                           |   4 +-
 src/box/box.h                            |   2 +-
 src/box/iproto.cc                        | 268 ++++++++++++++++++++++++------
 src/box/iproto_constants.h               |   3 +
 src/box/lua/call.c                       |   4 +-
 src/box/lua/console.c                    | 105 ++++++++++++
 src/box/lua/console.h                    |  10 ++
 src/box/lua/console.lua                  |  63 ++++---
 src/box/lua/net_box.lua                  |  50 ++++--
 src/box/lua/session.c                    | 116 ++++++++++++-
 src/box/port.c                           |  24 ++-
 src/box/port.h                           |  29 +++-
 src/box/session.cc                       |  50 +++++-
 src/box/session.h                        | 111 ++++++++++---
 src/box/xrow.c                           |  13 ++
 src/box/xrow.h                           |  12 ++
 src/fiber.h                              |  14 +-
 src/fio.c                                |   2 +-
 src/fio.h                                |   2 +-
 test/app-tap/console.test.lua            |  10 +-
 test/app-tap/yaml.test.lua               |  51 +++++-
 test/box/net.box.result                  |   4 +-
 test/box/net.box.test.lua                |   4 +-
 test/box/push.result                     | 275 +++++++++++++++++++++++++++++++
 test/box/push.test.lua                   | 147 +++++++++++++++++
 test/replication/before_replace.result   |  14 ++
 test/replication/before_replace.test.lua |  11 ++
 third_party/lua-yaml/lyaml.cc            | 185 +++++++++++++++++----
 third_party/lua-yaml/lyaml.h             |  29 ++++
 31 files changed, 1426 insertions(+), 193 deletions(-)
 create mode 100644 test/box/push.result
 create mode 100644 test/box/push.test.lua

-- 
2.15.1 (Apple Git-101)

             reply	other threads:[~2018-04-20 13:24 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-20 13:24 Vladislav Shpilevoy [this message]
2018-04-20 13:24 ` [PATCH v2 01/10] yaml: don't throw OOM on any error in yaml encoding Vladislav Shpilevoy
2018-05-10 18:10   ` [tarantool-patches] " Konstantin Osipov
2018-04-20 13:24 ` [tarantool-patches] [PATCH v2 10/10] session: introduce binary box.session.push Vladislav Shpilevoy
2018-05-10 19:50   ` Konstantin Osipov
2018-05-24 20:50     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-20 13:24 ` [PATCH v2 02/10] yaml: introduce yaml.encode_tagged Vladislav Shpilevoy
2018-05-10 18:22   ` [tarantool-patches] " Konstantin Osipov
2018-05-24 20:50     ` [tarantool-patches] " Vladislav Shpilevoy
2018-05-30 19:15       ` Konstantin Osipov
2018-05-30 20:49         ` Vladislav Shpilevoy
2018-05-31 10:46           ` Konstantin Osipov
2018-04-20 13:24 ` [PATCH v2 03/10] yaml: introduce yaml.decode_tag Vladislav Shpilevoy
2018-05-10 18:41   ` [tarantool-patches] " Konstantin Osipov
2018-05-24 20:50     ` [tarantool-patches] " Vladislav Shpilevoy
2018-05-31 10:54       ` Konstantin Osipov
2018-05-31 11:36       ` Konstantin Osipov
2018-04-20 13:24 ` [PATCH v2 04/10] console: use Lua C API to do formatting for console Vladislav Shpilevoy
2018-05-10 18:46   ` [tarantool-patches] " Konstantin Osipov
2018-05-24 20:50     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-20 13:24 ` [PATCH v2 05/10] session: move salt into iproto connection Vladislav Shpilevoy
2018-05-10 18:47   ` [tarantool-patches] " Konstantin Osipov
2018-04-20 13:24 ` [PATCH v2 06/10] session: introduce session vtab and meta Vladislav Shpilevoy
2018-05-10 19:20   ` [tarantool-patches] " Konstantin Osipov
2018-05-24 20:50     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-20 13:24 ` [PATCH v2 07/10] port: rename dump() into dump_msgpack() Vladislav Shpilevoy
2018-05-10 19:21   ` [tarantool-patches] " Konstantin Osipov
2018-04-20 13:24 ` [PATCH v2 08/10] session: introduce text box.session.push Vladislav Shpilevoy
2018-05-10 19:27   ` [tarantool-patches] " Konstantin Osipov
2018-05-24 20:50     ` [tarantool-patches] " Vladislav Shpilevoy
2018-04-20 13:24 ` [PATCH v2 09/10] session: enable box.session.push in local console Vladislav Shpilevoy
2018-05-10 19:28   ` [tarantool-patches] " Konstantin Osipov
2018-05-24 20:50 ` [tarantool-patches] [PATCH 1/1] netbox: introduce iterable future objects Vladislav Shpilevoy
2018-06-04 22:17   ` [tarantool-patches] " Vladislav Shpilevoy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1524228894.git.v.shpilevoy@tarantool.org \
    --to=v.shpilevoy@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --cc=vdavydov.dev@gmail.com \
    --subject='Re: [PATCH v2 00/10] session: introduce box.session.push' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox