From: Nikita Pettik <korablev@tarantool.org> To: Konstantin Osipov <kostja.osipov@gmail.com>, tarantool-patches@dev.tarantool.org, v.shpilevoy@tarantool.org Subject: Re: [Tarantool-patches] [PATCH 2/2] vinyl: skip invalid upserts during squash Date: Thu, 21 May 2020 02:51:54 +0000 [thread overview] Message-ID: <20200521025153.GC19031@tarantool.org> (raw) In-Reply-To: <20200519193948.GA12955@atlas> On 19 May 22:39, Konstantin Osipov wrote: > * Nikita Pettik <korablev@tarantool.org> [20/05/19 22:10]: > > On 14 May 09:56, Konstantin Osipov wrote: > > > * Nikita Pettik <korablev@tarantool.org> [20/05/14 09:50]: > > > > On 14 Apr 01:12, Konstantin Osipov wrote: > > > > > * Nikita Pettik <korablev@tarantool.org> [20/04/14 00:57]: > > > > > > > > Finally, let's double check there are no issues with the used > > > > > format - can it become obsolete by the time it's used, e.g. if > > > > > there is an online/non-blocking schema change that happened in tx > > > > > thread (compaction is running in the write thread)? > > > > > > > > Upserts are supported only by primary index. Meanwhile vinyl does > > > > not support altering primary index of non-empty space. Am I missing > > > > something? > > > > > > I mean the space format object itself. Squashing is happening in a > > > different thread, the > > > > > > > > Seems like you've cut sentence. Could you please suggest exact > > test scenario? We already have similar test in vinyl/errinj.test.lua: > > 1. Memtable dump is started. > 2. Space is altered. > 3. The format object is gone. > 4. Memtable dump uses the old format object for upsert > squashing. Look, the format is used for upsert squashing is stored in write iterator: in vy_read_view_merge() it is passed to vy_apply_upsert() from stream->format. When write iterator is created, format which is obtained (i.e. lsm->disk_format) is refed. So, as you can see, it can't be destroyed before write iterator is finished. Finally, as I already mentioned, lsm used for squashing is PK, and PK in turn can't be altered (lsm->disk_format will be destroyed only on pk:drop()). > Similar with compaction: > 1. Compaction starts and uses the existing format objects > 2. It uses the format object for upsert squashing > 3. Space is altered and format is changed > 4. We report a squash failure/suppress failure. > > > This is both related to memory liveness issues (e.g. we may use a > freed format object, which isn't the case because we refcount them > at start of dump/compaction), but also correctness issues: we > accept data which is no longer acceptable according to the new > format, or report/skip data which shouldn't be skipped. Data will always will be acceptable according to the format because PK format can't change. > -- > Konstantin Osipov, Moscow, Russia
next prev parent reply other threads:[~2020-05-21 2:51 UTC|newest] Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-04-13 21:55 [Tarantool-patches] [PATCH 0/2] Validate result of upserts squash & skip invalid upserts which can't be applied Nikita Pettik 2020-04-13 21:55 ` [Tarantool-patches] [PATCH 1/2] vinyl: validate resulting tuple after upsert is applied Nikita Pettik 2020-06-22 19:28 ` Aleksandr Lyapunov 2020-04-13 21:55 ` [Tarantool-patches] [PATCH 2/2] vinyl: skip invalid upserts during squash Nikita Pettik 2020-04-13 22:12 ` Konstantin Osipov 2020-05-14 2:11 ` Nikita Pettik 2020-05-14 6:56 ` Konstantin Osipov 2020-05-19 19:10 ` Nikita Pettik 2020-05-19 19:39 ` Konstantin Osipov 2020-05-21 2:51 ` Nikita Pettik [this message] 2020-05-21 8:36 ` Konstantin Osipov 2020-05-01 0:31 ` Vladislav Shpilevoy 2020-05-14 2:21 ` Nikita Pettik 2020-05-14 21:32 ` Vladislav Shpilevoy 2020-05-19 18:18 ` Nikita Pettik 2020-05-20 22:13 ` Vladislav Shpilevoy 2020-05-26 21:33 ` Vladislav Shpilevoy 2020-05-27 20:05 ` Nikita Pettik 2020-05-29 21:47 ` Vladislav Shpilevoy 2020-06-01 19:24 ` Nikita Pettik 2020-05-20 22:13 ` [Tarantool-patches] [PATCH 0/2] Validate result of upserts squash & skip invalid upserts which can't be applied Vladislav Shpilevoy 2020-05-22 2:42 ` Nikita Pettik 2020-05-26 21:33 ` Vladislav Shpilevoy 2020-05-27 20:10 ` Nikita Pettik 2020-06-22 14:13 ` Aleksandr Lyapunov 2020-06-22 20:21 ` Nikita Pettik 2020-06-23 12:32 ` Aleksandr Lyapunov 2020-06-02 21:36 ` Vladislav Shpilevoy 2020-06-02 21:37 ` Vladislav Shpilevoy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200521025153.GC19031@tarantool.org \ --to=korablev@tarantool.org \ --cc=kostja.osipov@gmail.com \ --cc=tarantool-patches@dev.tarantool.org \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH 2/2] vinyl: skip invalid upserts during squash' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox