Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladislav Shpilevoy via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: tarantool-patches@dev.tarantool.org, gorcunov@gmail.com,
	sergepetrenko@tarantool.org
Subject: [Tarantool-patches] [PATCH 00/16] Cord buffer, static alloc, and Lua GC bug
Date: Sat, 20 Mar 2021 01:42:30 +0100	[thread overview]
Message-ID: <cover.1616200860.git.v.shpilevoy@tarantool.org> (raw)

The patch attempts to fix most of the easy to face issues with the global
resources not having proper ownership in Lua code and therefore not protected
again being suddenly reused during Lua GC.

There are many such resources: tarantool_lua_ibuf/IBUF_SHARED, static alloc,
errno, diag/box.error, box_tuple_last, port_c, and others.

The most easy to screw - ibuf and static alloc. They are fixed in this
patchset.

The solution is dubious, but there were not found any better alternatives.

A patch for 1.10 will be sent later in a separate CL.

Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-5632-global-buf-crash-2x
Issue: https://github.com/tarantool/tarantool/issues/5632

Vladislav Shpilevoy (16):
  fio: don't use shared buffer in pread()
  test: don't use IBUF_SHARED in the tests
  tuple: pass global ibuf explicitly where possible
  iconv: take errno before reseting the context
  cord_buf: introduce cord_buf API
  cord_buf: introduce ownership management
  buffer: implement ffi stash
  uuid: replace static_alloc with ffi stash
  uuid: drop tt_uuid_str() from Lua
  uri: replace static_alloc with ffi stash and ibuf
  buffer: remove static_alloc() from Lua
  lua: use lua_pushfstring() instead of tt_sprintf()
  sio: rework sio_strfaddr()
  sio: increase SERVICE_NAME_MAXLEN size
  sio: introduce and use sio_snprintf()
  buffer: remove Lua registers

 changelogs/unreleased/fix-ibuf-static.md      |   7 +
 src/CMakeLists.txt                            |   1 -
 src/box/iproto.cc                             |   8 +-
 src/box/iproto.h                              |   2 +-
 src/box/lua/info.c                            |   5 +-
 src/box/lua/net_box.lua                       |   4 +-
 src/box/lua/schema.lua                        |  56 ++++--
 src/box/lua/session.c                         |   7 +-
 src/box/lua/space.cc                          |   6 +-
 src/box/lua/tuple.c                           |  80 +++++----
 src/box/lua/tuple.lua                         |  14 +-
 src/exports.h                                 |   7 +-
 src/lib/core/CMakeLists.txt                   |   1 +
 src/lib/core/cord_buf.c                       | 167 ++++++++++++++++++
 src/lib/core/cord_buf.h                       |  54 ++++++
 src/lib/core/sio.c                            |  72 ++++----
 src/lib/core/sio.h                            |  15 +-
 src/lua/buffer.c                              |  42 -----
 src/lua/buffer.lua                            | 156 +++++++++-------
 src/lua/crypto.lua                            |  19 +-
 src/lua/digest.lua                            |  25 ++-
 src/lua/fiber.c                               |   2 +-
 src/lua/fio.lua                               |  12 +-
 src/lua/iconv.lua                             |  27 +--
 src/lua/init.c                                |   3 -
 src/lua/msgpack.c                             |   6 +-
 src/lua/msgpackffi.lua                        |  57 +++---
 src/lua/socket.lua                            |  54 +++---
 src/lua/string.lua                            |  26 ++-
 src/lua/swim.lua                              |  19 +-
 src/lua/uri.lua                               |  21 ++-
 src/lua/utf8.c                                |  21 ++-
 src/lua/utils.h                               |   1 -
 src/lua/uuid.lua                              | 115 +++++++-----
 .../test/static-build/exports.test.lua        |   4 +-
 test/app-tap/buffer.test.lua                  |  59 +++++++
 test/app-tap/gh-5632-gc-buf-reuse.test.lua    | 151 ++++++++++++++++
 test/app-tap/module_api.test.lua              |  10 +-
 test/app-tap/uri.test.lua                     |  23 +--
 test/app/buffer.result                        |  53 ------
 test/app/buffer.test.lua                      |  23 ---
 test/app/fio.result                           | 160 +++++++++--------
 test/app/fio.test.lua                         |  67 ++++---
 test/app/msgpack.result                       |   7 +-
 test/app/msgpack.test.lua                     |   5 +-
 test/box/varbinary_type.result                |   3 +-
 test/box/varbinary_type.test.lua              |   3 +-
 test/unit/luaT_tuple_new.c                    |   4 -
 48 files changed, 1092 insertions(+), 592 deletions(-)
 create mode 100644 changelogs/unreleased/fix-ibuf-static.md
 create mode 100644 src/lib/core/cord_buf.c
 create mode 100644 src/lib/core/cord_buf.h
 delete mode 100644 src/lua/buffer.c
 create mode 100755 test/app-tap/buffer.test.lua
 create mode 100755 test/app-tap/gh-5632-gc-buf-reuse.test.lua
 delete mode 100644 test/app/buffer.result
 delete mode 100644 test/app/buffer.test.lua

-- 
2.24.3 (Apple Git-128)


             reply	other threads:[~2021-03-20  0:42 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-20  0:42 Vladislav Shpilevoy via Tarantool-patches [this message]
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 01/16] fio: don't use shared buffer in pread() Vladislav Shpilevoy via Tarantool-patches
2021-03-22  7:19   ` Cyrill Gorcunov via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 10/16] uri: replace static_alloc with ffi stash and ibuf Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 11/16] buffer: remove static_alloc() from Lua Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 12/16] lua: use lua_pushfstring() instead of tt_sprintf() Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 13/16] sio: rework sio_strfaddr() Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 14/16] sio: increase SERVICE_NAME_MAXLEN size Vladislav Shpilevoy via Tarantool-patches
2021-03-21 21:58   ` Cyrill Gorcunov via Tarantool-patches
2021-03-22 22:32     ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  6:56       ` Cyrill Gorcunov via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 15/16] sio: introduce and use sio_snprintf() Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 16/16] buffer: remove Lua registers Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 02/16] test: don't use IBUF_SHARED in the tests Vladislav Shpilevoy via Tarantool-patches
2021-03-22  7:35   ` Cyrill Gorcunov via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 03/16] tuple: pass global ibuf explicitly where possible Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 04/16] iconv: take errno before reseting the context Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 05/16] cord_buf: introduce cord_buf API Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 06/16] cord_buf: introduce ownership management Vladislav Shpilevoy via Tarantool-patches
2021-03-22 16:48   ` Serge Petrenko via Tarantool-patches
2021-03-22 22:32     ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  7:46       ` Serge Petrenko via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 07/16] buffer: implement ffi stash Vladislav Shpilevoy via Tarantool-patches
2021-03-23  0:29   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 08/16] uuid: replace static_alloc with " Vladislav Shpilevoy via Tarantool-patches
2021-03-20  0:42 ` [Tarantool-patches] [PATCH 09/16] uuid: drop tt_uuid_str() from Lua Vladislav Shpilevoy via Tarantool-patches
2021-03-21 16:38 ` [Tarantool-patches] [PATCH 00/16] Cord buffer, static alloc, and Lua GC bug Vladislav Shpilevoy via Tarantool-patches
2021-03-22  7:52 ` Cyrill Gorcunov via Tarantool-patches
2021-03-22  7:56 ` Konstantin Osipov via Tarantool-patches
2021-03-22 17:17 ` Serge Petrenko via Tarantool-patches
2021-03-23 23:45 ` Vladislav Shpilevoy via Tarantool-patches
2021-03-24 13:28 ` Kirill Yukhin via Tarantool-patches

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1616200860.git.v.shpilevoy@tarantool.org \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=gorcunov@gmail.com \
    --cc=sergepetrenko@tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH 00/16] Cord buffer, static alloc, and Lua GC bug' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox