From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladislav Shpilevoy Subject: [PATCH v2 00/10] session: introduce box.session.push Date: Fri, 20 Apr 2018 16:24:24 +0300 Message-Id: To: tarantool-patches@freelists.org Cc: vdavydov.dev@gmail.com List-ID: Branch: http://github.com/tarantool/tarantool/tree/gh-2677-box-session-push Issue: https://github.com/tarantool/tarantool/issues/2677 Sometimes it is needed to send some intermediate results from long poll request. Or to create a watcher on a server side, which listens for some events and sends notifications to a client. Or split a big response into multiple ones, as it is described in Tarantool Wire Protocol RFC. IPROTO_CHUNK protocol command solves all of the problems. IPROTO_CHUNK is a non-final response, and a single request can produce multiple response chunks. Box.session.push is API to send a chunk. It takes a Lua object and sends it to a client using the way depending of a session type: console or binary. Console session pushes are YAML documents tagged with using YAML %TAG feature - push message is tagged with !push! tag. For example: tarantool> box.session.push({a = 100, b = 200}) %TAG !push! tag:tarantool.io/push,2018 --- <--------- Here pushed message starts. a: 100 b: 200 ... --- <--------- Here push() call result starts. - true ... It is not the same as future console.print() pushes - they are not linked with chunks. Binary push is an IProto response with IPROTO_CHUNK key in the REQUEST_TYPE header field. In the rest it has the same structure as any another response. The most of the patchset work is preparative. First patches introduces YAML tags for Tarantool YAML codec library - they were implemented already in libyaml, but was not used in Tarantool. Next patches refactor session - virtual table and session meta depending on session type are added. Session meta is used by IProto sessions to store latest sync and connection, console session stores file descriptor of remote client, other sessions store nothing. Session meta is accessed via session vtable. IPROTO_CHUNK response must have the same sync as a request, and to avoid specifying it in the box.session.push() API, that is ugly, the request sync is stored in the local storage of the fiber that executes the request. Sync key does not extend the storage size - instead it shares one of storage cells with another fiber local variable, that is never used in IProto sessions. Vladislav Shpilevoy (10): yaml: don't throw OOM on any error in yaml encoding yaml: introduce yaml.encode_tagged yaml: introduce yaml.decode_tag console: use Lua C API to do formatting for console session: move salt into iproto connection session: introduce session vtab and meta port: rename dump() into dump_msgpack() session: introduce text box.session.push session: enable box.session.push in local console session: introduce binary box.session.push src/box/authentication.cc | 4 +- src/box/authentication.h | 3 +- src/box/box.cc | 4 +- src/box/box.h | 2 +- src/box/iproto.cc | 268 ++++++++++++++++++++++++------ src/box/iproto_constants.h | 3 + src/box/lua/call.c | 4 +- src/box/lua/console.c | 105 ++++++++++++ src/box/lua/console.h | 10 ++ src/box/lua/console.lua | 63 ++++--- src/box/lua/net_box.lua | 50 ++++-- src/box/lua/session.c | 116 ++++++++++++- src/box/port.c | 24 ++- src/box/port.h | 29 +++- src/box/session.cc | 50 +++++- src/box/session.h | 111 ++++++++++--- src/box/xrow.c | 13 ++ src/box/xrow.h | 12 ++ src/fiber.h | 14 +- src/fio.c | 2 +- src/fio.h | 2 +- test/app-tap/console.test.lua | 10 +- test/app-tap/yaml.test.lua | 51 +++++- test/box/net.box.result | 4 +- test/box/net.box.test.lua | 4 +- test/box/push.result | 275 +++++++++++++++++++++++++++++++ test/box/push.test.lua | 147 +++++++++++++++++ test/replication/before_replace.result | 14 ++ test/replication/before_replace.test.lua | 11 ++ third_party/lua-yaml/lyaml.cc | 185 +++++++++++++++++---- third_party/lua-yaml/lyaml.h | 29 ++++ 31 files changed, 1426 insertions(+), 193 deletions(-) create mode 100644 test/box/push.result create mode 100644 test/box/push.test.lua -- 2.15.1 (Apple Git-101)