<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">22 мая 2019 г., в 19:28, Vladimir Davydov <<a href="mailto:vdavydov.dev@gmail.com" class="">vdavydov.dev@gmail.com</a>> написал(а):</div><br class="Apple-interchange-newline"><div class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">On Wed, May 22, 2019 at 07:11:42PM +0300, Serge Petrenko wrote:</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">Memtx index build used to stall event loop for all the build period.<br class="">Add occasional yields so that the loop is not blocked for too long.<br class=""><br class="">Closes #3976<br class="">---<br class=""><a href="https://github.com/tarantool/tarantool/issues/3976" class="">https://github.com/tarantool/tarantool/issues/3976</a><br class="">https://github.com/tarantool/tarantool/tree/sp/gh-3976-background-index-build<br class=""><br class="">src/box/memtx_space.c | 13 +++++<br class="">test/box/memtx_background_index_build.result | 55 +++++++++++++++++++<br class="">.../box/memtx_background_index_build.test.lua | 32 +++++++++++<br class="">test/box/suite.ini | 2 +-<br class="">4 files changed, 101 insertions(+), 1 deletion(-)<br class="">create mode 100644 test/box/memtx_background_index_build.result<br class="">create mode 100644 test/box/memtx_background_index_build.test.lua<br class=""><br class="">diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c<br class="">index 5ddb4f7ee..b90e2707e 100644<br class="">--- a/src/box/memtx_space.c<br class="">+++ b/src/box/memtx_space.c<br class="">@@ -874,6 +874,15 @@ static int<br class="">memtx_space_build_index(struct space *src_space, struct index *new_index,<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>struct tuple_format *new_format)<br class="">{<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/*<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* Yield every 1K tuples.<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* In debug mode yield more often for testing purposes.<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>*/<br class="">+#ifdef NDEBUG<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>enum { YIELD_LOOPS = 1000 };<br class="">+#else<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>enum { YIELD_LOOPS = 10 };<br class="">+#endif<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>/**<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* If it's a secondary key, and we're not building them<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* yet (i.e. it's snapshot recovery for memtx), do nothing.<br class="">@@ -909,6 +918,7 @@ memtx_space_build_index(struct space *src_space, struct index *new_index,<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>/* Build the new index. */<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>int rc;<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>struct tuple *tuple;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>size_t count = 0;<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>while ((rc = iterator_next(it, &tuple)) == 0 && tuple != NULL) {<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/*<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* Check that the tuple is OK according to the<br class="">@@ -933,6 +943,9 @@ memtx_space_build_index(struct space *src_space, struct index *new_index,<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>*/<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (new_index->def->iid == 0)<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>tuple_ref(tuple);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (++count % YIELD_LOOPS == 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>fiber_sleep(0);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class=""></blockquote><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">This isn't enough as tuples may be inserted into / deleted from the</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">space while DDL is in progress. We need to propagate these changes</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">to the new index using on_replace trigger. Take a look at how vinyl</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">handles it.</span></div></blockquote></div><br class=""><div class="">Thank you for pointing this out! Please see v2 in the mailing list.</div></body></html>