From: Vladislav Shpilevoy <v.shpilevoy@tarantool.org> To: tarantool-patches@freelists.org Subject: [tarantool-patches] [PATCH 00/13] JSON updates Date: Tue, 13 Aug 2019 01:05:06 +0200 [thread overview] Message-ID: <cover.1565649886.git.v.shpilevoy@tarantool.org> (raw) The patchset introduces JSON path updates. Enough words have already been said - this feature is obviously needed, and community is desperate to be able to update tuple internals. Since the preparatory patchset still is not pushed, it is included here. The patchset is already huge. And nonetheless there is still some work to be done as follow-ups. Perhaps, the points below are worth reading only after you took a look at the patchset. Otherwise you won't understand some points. - Single update operation can be optimized to not even create a rope. Bar update can easily handle that; - Bar update can store a pointer not to only the updated point, but to its parent map/array too. It can strongly speed up one common case of JSON updates, when they differ in the last JSON token only. - Perhaps, route updates could store a pointer to MessagePack for each JSON token in the route's path. It would allow to use the existing column mask to check which updates intersect, and for them create routes instead of bars. It would eliminate extra decoding of MessagePack. - The implementation is recursive, but perhaps it is worth rewriting to a non-recursive version with keeping intermediate results on a region. Not sure though, if it is worth doing, because there is usually a very few update operations, and stack can fit an update tree of 4k depth easily - it is our current limit on update operation count. 4k is unreachable anyway. All knows cases are <<50 operations. Usually <= 5. - struct update_field and update_op should be stored in a pool instead of region. They were quite big even before my patchset, and there is no any reason why they need to be on a region. Here they consume transaction memory. I will create issues for these things if this patchset is pushed someday. Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-1261-update-json Issue: https://github.com/tarantool/tarantool/issues/1261 Vladislav Shpilevoy (13): ----------------------- Preparatory patches. It is some refactoring to simplify further updates code rework, to speed it up, to introduce a teaser-feature - updates by field names. tuple: remove alloc and alloc_ctx args from update() rope: make rope library macro template tuple: relax struct tuple_update dependency on rope int96: add a missing header tuple: implement update by field name tuple: expose JSON go_to_key and go_to_index functions ----------------------- First patch relates to JSON updates - it reorganizes code into separate files for each update type. Similar to some other implementations I've seen. Others are just minor independent refactorings. tuple: rework updates to improve code extendibility json: lexer_eof and token_cmp helper functions tuple: account the whole array in field.data and size ----------------------- The core part of the patch. JSON updates are split into 3 independent parts: non-intersecting update paths, array updates, map updates. tuple: enable JSON bar updates tuple: make update operation tokens consumable tuple: JSON updates support intersection by arrays tuple: JSON updates support intersection by maps debian/copyright | 2 +- src/box/CMakeLists.txt | 5 + src/box/errcode.h | 12 +- src/box/lua/tuple.c | 10 +- src/box/memtx_space.c | 27 +- src/box/space.c | 26 +- src/box/sql/insert.c | 3 +- src/box/sql/resolve.c | 4 +- src/box/sql/update.c | 2 +- src/box/tuple.c | 39 +- src/box/tuple.h | 23 + src/box/tuple_update.c | 1098 +++------------------------------ src/box/tuple_update.h | 25 +- src/box/update/update_array.c | 394 ++++++++++++ src/box/update/update_bar.c | 407 ++++++++++++ src/box/update/update_field.c | 630 +++++++++++++++++++ src/box/update/update_field.h | 707 +++++++++++++++++++++ src/box/update/update_map.c | 441 +++++++++++++ src/box/update/update_route.c | 374 +++++++++++ src/box/vinyl.c | 36 +- src/box/vy_upsert.c | 31 +- src/lib/bit/int96.h | 1 + src/lib/json/json.c | 37 +- src/lib/json/json.h | 28 + src/lib/salad/rope.c | 479 +++----------- src/lib/salad/rope.h | 617 +++++++++++++++--- test/box/misc.result | 5 +- test/box/update.result | 739 +++++++++++++++++++++- test/box/update.test.lua | 311 ++++++++++ test/engine/update.result | 62 ++ test/engine/update.test.lua | 26 + test/engine/upsert.result | 2 +- test/unit/column_mask.c | 33 +- test/unit/rope.c | 1 - test/unit/rope_avl.c | 1 - test/unit/rope_basic.c | 1 - test/unit/rope_common.h | 41 +- test/unit/rope_stress.c | 5 +- 38 files changed, 5004 insertions(+), 1681 deletions(-) create mode 100644 src/box/update/update_array.c create mode 100644 src/box/update/update_bar.c create mode 100644 src/box/update/update_field.c create mode 100644 src/box/update/update_field.h create mode 100644 src/box/update/update_map.c create mode 100644 src/box/update/update_route.c -- 2.20.1 (Apple Git-117)
next reply other threads:[~2019-08-12 23:02 UTC|newest] Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-08-12 23:05 Vladislav Shpilevoy [this message] 2019-08-12 23:05 ` [tarantool-patches] [PATCH 01/13] tuple: remove alloc and alloc_ctx args from update() Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 10/13] tuple: enable JSON bar updates Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 11/13] tuple: make update operation tokens consumable Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 12/13] tuple: JSON updates support intersection by arrays Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 13/13] tuple: JSON updates support intersection by maps Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 02/13] rope: make rope library macro template Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 03/13] tuple: relax struct tuple_update dependency on rope Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 04/13] int96: add a missing header Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 05/13] tuple: implement update by field name Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 06/13] tuple: expose JSON go_to_key and go_to_index functions Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 07/13] tuple: rework updates to improve code extendibility Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 08/13] json: lexer_eof and token_cmp helper functions Vladislav Shpilevoy 2019-08-12 23:05 ` [tarantool-patches] [PATCH 09/13] tuple: account the whole array in field.data and size Vladislav Shpilevoy 2019-08-20 18:49 ` [tarantool-patches] Re: [PATCH 00/13] JSON updates Vladislav Shpilevoy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cover.1565649886.git.v.shpilevoy@tarantool.org \ --to=v.shpilevoy@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [tarantool-patches] [PATCH 00/13] JSON updates' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox