[tarantool-patches] [PATCH 00/13] JSON updates
Vladislav Shpilevoy
v.shpilevoy at tarantool.org
Tue Aug 13 02:05:06 MSK 2019
The patchset introduces JSON path updates. Enough words have already been said -
this feature is obviously needed, and community is desperate to be able to
update tuple internals.
Since the preparatory patchset still is not pushed, it is included here.
The patchset is already huge. And nonetheless there is still some work to be
done as follow-ups. Perhaps, the points below are worth reading only after you
took a look at the patchset. Otherwise you won't understand some points.
- Single update operation can be optimized to not even create a rope. Bar update
can easily handle that;
- Bar update can store a pointer not to only the updated point, but to its
parent map/array too. It can strongly speed up one common case of JSON
updates, when they differ in the last JSON token only.
- Perhaps, route updates could store a pointer to MessagePack for each JSON
token in the route's path. It would allow to use the existing column mask to
check which updates intersect, and for them create routes instead of bars. It
would eliminate extra decoding of MessagePack.
- The implementation is recursive, but perhaps it is worth rewriting to a
non-recursive version with keeping intermediate results on a region. Not sure
though, if it is worth doing, because there is usually a very few update
operations, and stack can fit an update tree of 4k depth easily - it is our
current limit on update operation count. 4k is unreachable anyway. All knows
cases are <<50 operations. Usually <= 5.
- struct update_field and update_op should be stored in a pool instead of
region. They were quite big even before my patchset, and there is no any
reason why they need to be on a region. Here they consume transaction memory.
I will create issues for these things if this patchset is pushed someday.
Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-1261-update-json
Issue: https://github.com/tarantool/tarantool/issues/1261
Vladislav Shpilevoy (13):
-----------------------
Preparatory patches. It is some refactoring to simplify further updates code
rework, to speed it up, to introduce a teaser-feature - updates by field names.
tuple: remove alloc and alloc_ctx args from update()
rope: make rope library macro template
tuple: relax struct tuple_update dependency on rope
int96: add a missing header
tuple: implement update by field name
tuple: expose JSON go_to_key and go_to_index functions
-----------------------
First patch relates to JSON updates - it reorganizes code into separate files
for each update type. Similar to some other implementations I've seen. Others
are just minor independent refactorings.
tuple: rework updates to improve code extendibility
json: lexer_eof and token_cmp helper functions
tuple: account the whole array in field.data and size
-----------------------
The core part of the patch. JSON updates are split into 3 independent parts:
non-intersecting update paths, array updates, map updates.
tuple: enable JSON bar updates
tuple: make update operation tokens consumable
tuple: JSON updates support intersection by arrays
tuple: JSON updates support intersection by maps
debian/copyright | 2 +-
src/box/CMakeLists.txt | 5 +
src/box/errcode.h | 12 +-
src/box/lua/tuple.c | 10 +-
src/box/memtx_space.c | 27 +-
src/box/space.c | 26 +-
src/box/sql/insert.c | 3 +-
src/box/sql/resolve.c | 4 +-
src/box/sql/update.c | 2 +-
src/box/tuple.c | 39 +-
src/box/tuple.h | 23 +
src/box/tuple_update.c | 1098 +++------------------------------
src/box/tuple_update.h | 25 +-
src/box/update/update_array.c | 394 ++++++++++++
src/box/update/update_bar.c | 407 ++++++++++++
src/box/update/update_field.c | 630 +++++++++++++++++++
src/box/update/update_field.h | 707 +++++++++++++++++++++
src/box/update/update_map.c | 441 +++++++++++++
src/box/update/update_route.c | 374 +++++++++++
src/box/vinyl.c | 36 +-
src/box/vy_upsert.c | 31 +-
src/lib/bit/int96.h | 1 +
src/lib/json/json.c | 37 +-
src/lib/json/json.h | 28 +
src/lib/salad/rope.c | 479 +++-----------
src/lib/salad/rope.h | 617 +++++++++++++++---
test/box/misc.result | 5 +-
test/box/update.result | 739 +++++++++++++++++++++-
test/box/update.test.lua | 311 ++++++++++
test/engine/update.result | 62 ++
test/engine/update.test.lua | 26 +
test/engine/upsert.result | 2 +-
test/unit/column_mask.c | 33 +-
test/unit/rope.c | 1 -
test/unit/rope_avl.c | 1 -
test/unit/rope_basic.c | 1 -
test/unit/rope_common.h | 41 +-
test/unit/rope_stress.c | 5 +-
38 files changed, 5004 insertions(+), 1681 deletions(-)
create mode 100644 src/box/update/update_array.c
create mode 100644 src/box/update/update_bar.c
create mode 100644 src/box/update/update_field.c
create mode 100644 src/box/update/update_field.h
create mode 100644 src/box/update/update_map.c
create mode 100644 src/box/update/update_route.c
--
2.20.1 (Apple Git-117)
More information about the Tarantool-patches
mailing list