From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 68FBB25BD3 for ; Sat, 31 Aug 2019 17:32:41 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p4zVq5SjgnlH for ; Sat, 31 Aug 2019 17:32:41 -0400 (EDT) Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id AE39625BA5 for ; Sat, 31 Aug 2019 17:32:40 -0400 (EDT) From: Vladislav Shpilevoy Subject: [tarantool-patches] [PATCH v2 0/8] JSON updates Date: Sat, 31 Aug 2019 23:35:50 +0200 Message-Id: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: kostja@tarantool.org The patchset introduces JSON path updates. Enough words have already been said - this feature is obviously needed, and community is desperate to be able to update tuple internals. The patchset is already huge. And nonetheless there is still some work to be done as follow-ups. Perhaps, the points below are worth reading only after you took a look at the patchset. Otherwise you won't understand some points. - Single update operation can be optimized to not even create a rope. Bar update can easily handle that; - Bar update can store a pointer not to only the updated point, but to its parent map/array too. It can strongly speed up one common case of JSON updates, when they differ in the last JSON token only. - Perhaps, route updates could store a pointer to MessagePack for each JSON token in the route's path. It would allow to use the existing column mask to check which updates intersect, and for them create routes instead of bars. It would eliminate extra decoding of MessagePack. - The implementation is recursive, but perhaps it is worth rewriting to a non-recursive version with keeping intermediate results on a region. Not sure though, if it is worth doing, because there is usually a very few update operations, and stack can fit an update tree of 4k depth easily - it is our current limit on update operation count. 4k is unreachable anyway. All knows cases are <<50 operations. Usually <= 5. - struct update_field and update_op should be stored in a pool instead of region. They were quite big even before my patchset, and there is no any reason why they need to be on a region. Here they consume transaction memory. I will create issues for these things if this patchset is pushed someday. Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-1261-update-json Issue: https://github.com/tarantool/tarantool/issues/1261 Changes in V2: - Fixed a bug with wrong calculation of column mask for updates with path; - Preparatory patchset is pushed; - Rebase; I omit links to freelist. Anyway nobody reads old versions. Vladislav Shpilevoy (8): tuple: expose JSON go_to_key and go_to_index functions tuple: rework updates to improve code extendibility json: lexer_eof and token_cmp helper functions tuple: account the whole array in field.data and size tuple: enable JSON bar updates tuple: make update operation tokens consumable tuple: JSON updates support intersection by arrays tuple: JSON updates support intersection by maps src/box/CMakeLists.txt | 5 + src/box/errcode.h | 10 +- src/box/tuple.c | 21 +- src/box/tuple.h | 23 + src/box/tuple_update.c | 1178 +++------------------------------ src/box/update/update_array.c | 394 +++++++++++ src/box/update/update_bar.c | 407 ++++++++++++ src/box/update/update_field.c | 706 ++++++++++++++++++++ src/box/update/update_field.h | 710 ++++++++++++++++++++ src/box/update/update_map.c | 441 ++++++++++++ src/box/update/update_route.c | 374 +++++++++++ src/box/vinyl.c | 17 +- src/lib/json/json.c | 37 +- src/lib/json/json.h | 28 + test/box/misc.result | 2 +- test/box/tuple.result | 4 +- test/box/update.result | 739 ++++++++++++++++++++- test/box/update.test.lua | 311 +++++++++ test/engine/update.result | 5 - test/engine/update.test.lua | 2 - test/unit/column_mask.c | 75 ++- test/unit/column_mask.result | 8 +- 22 files changed, 4339 insertions(+), 1158 deletions(-) create mode 100644 src/box/update/update_array.c create mode 100644 src/box/update/update_bar.c create mode 100644 src/box/update/update_field.c create mode 100644 src/box/update/update_field.h create mode 100644 src/box/update/update_map.c create mode 100644 src/box/update/update_route.c -- 2.20.1 (Apple Git-117)