Tarantool development patches archive
 help / color / mirror / Atom feed
* [tarantool-patches] [PATCH v2 0/8] JSON updates
@ 2019-08-31 21:35 Vladislav Shpilevoy
  2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 1/8] tuple: expose JSON go_to_key and go_to_index functions Vladislav Shpilevoy
                   ` (7 more replies)
  0 siblings, 8 replies; 11+ messages in thread
From: Vladislav Shpilevoy @ 2019-08-31 21:35 UTC (permalink / raw)
  To: tarantool-patches; +Cc: kostja

The patchset introduces JSON path updates. Enough words have already been said -
this feature is obviously needed, and community is desperate to be able to
update tuple internals.

The patchset is already huge. And nonetheless there is still some work to be
done as follow-ups. Perhaps, the points below are worth reading only after you
took a look at the patchset. Otherwise you won't understand some points.

- Single update operation can be optimized to not even create a rope. Bar update
  can easily handle that;

- Bar update can store a pointer not to only the updated point, but to its
  parent map/array too. It can strongly speed up one common case of JSON
  updates, when they differ in the last JSON token only.

- Perhaps, route updates could store a pointer to MessagePack for each JSON
  token in the route's path. It would allow to use the existing column mask to
  check which updates intersect, and for them create routes instead of bars. It
  would eliminate extra decoding of MessagePack.

- The implementation is recursive, but perhaps it is worth rewriting to a
  non-recursive version with keeping intermediate results on a region. Not sure
  though, if it is worth doing, because there is usually a very few update
  operations, and stack can fit an update tree of 4k depth easily - it is our
  current limit on update operation count. 4k is unreachable anyway. All knows
  cases are <<50 operations. Usually <= 5.

- struct update_field and update_op should be stored in a pool instead of
  region. They were quite big even before my patchset, and there is no any
  reason why they need to be on a region. Here they consume transaction memory.

I will create issues for these things if this patchset is pushed someday.

Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-1261-update-json
Issue: https://github.com/tarantool/tarantool/issues/1261

Changes in V2:
- Fixed a bug with wrong calculation of column mask for updates with path;
- Preparatory patchset is pushed;
- Rebase;

I omit links to freelist. Anyway nobody reads old versions.

Vladislav Shpilevoy (8):
  tuple: expose JSON go_to_key and go_to_index functions
  tuple: rework updates to improve code extendibility
  json: lexer_eof and token_cmp helper functions
  tuple: account the whole array in field.data and size
  tuple: enable JSON bar updates
  tuple: make update operation tokens consumable
  tuple: JSON updates support intersection by arrays
  tuple: JSON updates support intersection by maps

 src/box/CMakeLists.txt        |    5 +
 src/box/errcode.h             |   10 +-
 src/box/tuple.c               |   21 +-
 src/box/tuple.h               |   23 +
 src/box/tuple_update.c        | 1178 +++------------------------------
 src/box/update/update_array.c |  394 +++++++++++
 src/box/update/update_bar.c   |  407 ++++++++++++
 src/box/update/update_field.c |  706 ++++++++++++++++++++
 src/box/update/update_field.h |  710 ++++++++++++++++++++
 src/box/update/update_map.c   |  441 ++++++++++++
 src/box/update/update_route.c |  374 +++++++++++
 src/box/vinyl.c               |   17 +-
 src/lib/json/json.c           |   37 +-
 src/lib/json/json.h           |   28 +
 test/box/misc.result          |    2 +-
 test/box/tuple.result         |    4 +-
 test/box/update.result        |  739 ++++++++++++++++++++-
 test/box/update.test.lua      |  311 +++++++++
 test/engine/update.result     |    5 -
 test/engine/update.test.lua   |    2 -
 test/unit/column_mask.c       |   75 ++-
 test/unit/column_mask.result  |    8 +-
 22 files changed, 4339 insertions(+), 1158 deletions(-)
 create mode 100644 src/box/update/update_array.c
 create mode 100644 src/box/update/update_bar.c
 create mode 100644 src/box/update/update_field.c
 create mode 100644 src/box/update/update_field.h
 create mode 100644 src/box/update/update_map.c
 create mode 100644 src/box/update/update_route.c

-- 
2.20.1 (Apple Git-117)

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2019-10-19 15:06 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-31 21:35 [tarantool-patches] [PATCH v2 0/8] JSON updates Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 1/8] tuple: expose JSON go_to_key and go_to_index functions Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 2/8] tuple: rework updates to improve code extendibility Vladislav Shpilevoy
     [not found]   ` <20190903192059.GE15611@atlas>
     [not found]     ` <6ee759cf-a975-e6a9-6f52-f855958ffe06@tarantool.org>
     [not found]       ` <20191005132055.GD3913@atlas>
     [not found]         ` <20191005135037.GJ3913@atlas>
2019-10-19 15:11           ` [Tarantool-patches] [tarantool-patches] " Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 3/8] json: lexer_eof and token_cmp helper functions Vladislav Shpilevoy
     [not found]   ` <20190903192433.GF15611@atlas>
     [not found]     ` <f5612e04-dc56-f4bd-1298-c5841ac909f5@tarantool.org>
     [not found]       ` <20191005132231.GE3913@atlas>
     [not found]         ` <20191005135014.GI3913@atlas>
2019-10-19 15:08           ` [Tarantool-patches] [tarantool-patches] " Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 4/8] tuple: account the whole array in field.data and size Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 5/8] tuple: enable JSON bar updates Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 6/8] tuple: make update operation tokens consumable Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 7/8] tuple: JSON updates support intersection by arrays Vladislav Shpilevoy
2019-08-31 21:35 ` [tarantool-patches] [PATCH v2 8/8] tuple: JSON updates support intersection by maps Vladislav Shpilevoy

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox