From: Konstantin Osipov <kostja.osipov@gmail.com> To: Nikita Pettik <korablev@tarantool.org> Cc: tarantool-patches@dev.tarantool.org, v.shpilevoy@tarantool.org Subject: Re: [Tarantool-patches] [PATCH v2] vinyl: restart read iterator in case of rolled back WAL Date: Fri, 19 Jun 2020 15:42:29 +0300 [thread overview] Message-ID: <20200619124229.GB61079@atlas> (raw) In-Reply-To: <20200619122412.GA19725@tarantool.org> * Nikita Pettik <korablev@tarantool.org> [20/06/19 15:26]: > On 16 Jun 15:10, Aleksandr Lyapunov wrote: > > Thanks for the patch! See my 2 comments below: > > > > On 6/1/20 7:46 PM, Nikita Pettik wrote: > > > + if (vy_mem_tree_iterator_is_invalid(&src->mem_iterator.curr_pos)) { > > > + assert(src->mem_iterator.curr.stmt == NULL); > > > + return 1; > > > + } > > I'm afraid that the iterator will not always be invalid in the given case. > > As I see, if a mem holds any older tuple with the same key (e.g. older > > version of that tuple), the restoration routine will find the older tuple > > with the non-invalid iterator. > > I also think that mem_restore must handle all the possible data > > changes by itself without concern of read_iterator. > > You are likely to be right, but I followed suggestion below > to simplify resotration procedure. > > > > - if (vy_read_iterator_restore_mem(itr, &next) != 0) > > > + int rc = vy_read_iterator_restore_mem(itr, &next); > > > + if (rc < 0) > > > return -1; > > > + if (rc > 0) { > > > + vy_read_iterator_restore(itr); > > > + goto restart; > > > + } > > The read iterator was rewritten several times and still have at least > > several bugs. I think we should admit that we cannot support such a > > complicated solution. How about some stupid solution: if ANY change > > has been happened during yield - restart advancing? Does this statement have technical merit? Is it supported by tests? I'd gladly support the change if it was grounded in reason - evaluation of the performance impact, for example, could serve as a confirmation that a simple solution would be just fine. Without it, I'd it's a regress, a signature of helplessness, lack of courage to make things right. > > diff --git a/src/box/vy_read_iterator.c b/src/box/vy_read_iterator.c > index 62a8722d9..409796910 100644 > --- a/src/box/vy_read_iterator.c > +++ b/src/box/vy_read_iterator.c > @@ -378,65 +378,6 @@ vy_read_iterator_scan_disk(struct vy_read_iterator *itr, uint32_t disk_src, > return 0; > } > > -/** > - * Restore the position of the active in-memory tree iterator > - * after a yield caused by a disk read and update 'next' > - * if necessary. > - * @retval -1 In case of error (e.g. OOM); > - * @retval 0 Successful execution; > - * @retval 1 Restart of advance_iterator is required. > - */ > -static NODISCARD int > -vy_read_iterator_restore_mem(struct vy_read_iterator *itr, > - struct vy_entry *next) > -{ > - int rc; > - int cmp; > - struct vy_read_src *src = &itr->src[itr->mem_src]; > - > - rc = vy_mem_iterator_restore(&src->mem_iterator, > - itr->last, &src->history); > - if (rc < 0) > - return -1; /* memory allocation error */ > - if (rc == 0) > - return 0; /* nothing changed */ > - > - if (vy_mem_tree_iterator_is_invalid(&src->mem_iterator.curr_pos)) { > - assert(src->mem_iterator.curr.stmt == NULL); > - return 1; > - } > - struct vy_entry entry = vy_history_last_stmt(&src->history); > - cmp = vy_read_iterator_cmp_stmt(itr, entry, *next); > - if (cmp > 0) { > - /* > - * Memory trees are append-only so if the > - * source is not on top of the heap after > - * restoration, it was not before. > - */ > - assert(src->front_id < itr->front_id); > - return 0; > - } > - if (cmp < 0) { > - /* > - * The new statement precedes the current > - * candidate for the next key. > - */ > - *next = entry; > - itr->front_id++; > - } else { > - /* > - * The new statement updates the next key. > - * Make sure we don't read the old value > - * from the cache while applying UPSERTs. > - */ > - struct vy_read_src *cache_src = &itr->src[itr->cache_src]; > - if (cache_src->front_id == itr->front_id) > - vy_history_cleanup(&cache_src->history); > - } > - src->front_id = itr->front_id; > - return 0; > -} > - > static void > vy_read_iterator_restore(struct vy_read_iterator *itr); > > @@ -536,10 +477,8 @@ rescan_disk: > * as it is owned exclusively by the current fiber so the only > * source to check is the active in-memory tree. > */ > - int rc = vy_read_iterator_restore_mem(itr, &next); > - if (rc < 0) > - return -1; > - if (rc > 0) { > + struct vy_mem_iterator *mem_itr = &itr->src[itr->mem_src].mem_iterator; > + if (mem_itr->version != mem_itr->mem->version) { > vy_read_iterator_restore(itr); > goto restart; > } > -- Konstantin Osipov, Moscow, Russia
next prev parent reply other threads:[~2020-06-19 12:42 UTC|newest] Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-01 16:46 Nikita Pettik 2020-06-01 17:40 ` Vladislav Shpilevoy 2020-06-19 12:34 ` Nikita Pettik 2020-06-20 16:33 ` Vladislav Shpilevoy 2020-06-16 12:10 ` Aleksandr Lyapunov 2020-06-19 12:24 ` Nikita Pettik 2020-06-19 12:42 ` Konstantin Osipov [this message] 2020-06-23 5:15 ` Aleksandr Lyapunov 2020-06-23 11:08 ` Konstantin Osipov 2020-06-19 13:01 ` Aleksandr Lyapunov 2020-06-24 13:41 ` Nikita Pettik 2020-06-20 16:33 ` Vladislav Shpilevoy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200619124229.GB61079@atlas \ --to=kostja.osipov@gmail.com \ --cc=korablev@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v2] vinyl: restart read iterator in case of rolled back WAL' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox