From v.shpilevoy at tarantool.org Sat Aug 1 01:31:30 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Sat, 1 Aug 2020 00:31:30 +0200 Subject: [Tarantool-patches] [PATCH 0/3] Qsync tricky crashes In-Reply-To: <20200731135913.GI3264@grain> References: <20200731135913.GI3264@grain> Message-ID: <766b8cfc-05e9-871f-3091-f4f1679275cb@tarantool.org> Pushed to master and 2.5. On 31.07.2020 15:59, Cyrill Gorcunov wrote: > On Fri, Jul 31, 2020 at 12:37:42AM +0200, Vladislav Shpilevoy wrote: >> The patchset fixes 2 crashes in qsync. 5185 wasn't reproduced, but a similar >> test gave another crash, fixed here. >> >> Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-5185-5195-qsync-crashes >> Issue: https://github.com/tarantool/tarantool/issues/5185 >> Issue: https://github.com/tarantool/tarantool/issues/5195 >> >> @ChangeLog >> * Fixed a crash when replica cleared synchronous transaction queue, while it was not empty on master (gh-5195). >> * Fixed a crash when synchronous transaction rollback and confirm could be written simultaneously for the same LSN (gh-5185). > > Ack. Lets give them a spin. > From v.shpilevoy at tarantool.org Sun Aug 2 17:44:28 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Sun, 2 Aug 2020 16:44:28 +0200 Subject: [Tarantool-patches] [PATCH 2/2] vinyl: rework upsert operation In-Reply-To: <14aec86329c7820470e40e47a3d1b5655b531732.1595985135.git.korablev@tarantool.org> References: <14aec86329c7820470e40e47a3d1b5655b531732.1595985135.git.korablev@tarantool.org> Message-ID: <444ff8e3-d6c2-e6a5-8b54-18a521208ba5@tarantool.org> Thanks for the patch! ASAN tests on the branch are failing: https://gitlab.com/tarantool/tarantool/-/jobs/661977877 See 19 comments below. > src/box/vinyl.c | 2 +- > src/box/vy_stmt.c | 28 ++-- > src/box/vy_stmt.h | 5 +- > src/box/vy_upsert.c | 305 +++++++++++++++++++++++++++------------- > test/unit/vy_iterators_helper.c | 2 +- > test/vinyl/upsert.result | 289 +++++++++++++++++++++++++++++++++++++ > test/vinyl/upsert.test.lua | 121 ++++++++++++++++ > 7 files changed, 644 insertions(+), 108 deletions(-) > > diff --git a/src/box/vy_upsert.c b/src/box/vy_upsert.c > index 797492c2b..caf2482c7 100644 > --- a/src/box/vy_upsert.c > +++ b/src/box/vy_upsert.c > @@ -68,12 +68,173 @@ vy_upsert_try_to_squash(struct tuple_format *format, > operations[0].iov_len = squashed_size; > > *result_stmt = vy_stmt_new_upsert(format, key_mp, key_mp_end, > - operations, 1); > + operations, 1, false); > if (*result_stmt == NULL) > return -1; > return 0; > } > > +/** > + * Check that key hasn't been changed after applying upsert operation. > + */ > +static bool > +vy_apply_result_does_cross_pk(struct tuple *old_stmt, const char *result, > + const char *result_end, struct key_def *cmp_def, > + uint64_t col_mask) > +{ > + if (!key_update_can_be_skipped(cmp_def->column_mask, col_mask)) { > + struct tuple *tuple = > + vy_stmt_new_replace(tuple_format(old_stmt), result, > + result_end); > + int cmp_res = vy_stmt_compare(old_stmt, HINT_NONE, tuple, > + HINT_NONE, cmp_def); 1. Bad indentation. > + tuple_unref(tuple); > + return cmp_res != 0; > + } > + return false; > +} > + > +/** > + * Apply update operations stored in @new_stmt (which is assumed to 2. Please, look at the doxygen syntax on the official page or here: https://github.com/tarantool/tarantool/wiki/Code-review-procedure This is a single very simple rule I keep repeating I don't already remember how many times - use @a , not @. I don't understand why does everyone keep violating it. 2. Parameter 'new_stmt' does not exist. As well as 'old_stmt'. What did you mean? > + * be upsert statement) on tuple @old_stmt. If @old_stmt is void > + * statement (i.e. it is NULL or delete statement) then operations > + * are applied on tuple @new_stmt. All operations which can't be > + * applied are skipped; errors may be logged depending on @supress_error 3. supress_error -> supress_error. 4. What do you mean as 'all operations'? Operation groups from various upserts? Or individual operations? > + * flag. > + * > + * @upsert Upsert statement to be applied on @stmt. 5. If you want to use doxygen, use @param . > + * @stmt Statement to be used as base for upsert operations. > + * @cmp_def Key definition required to provide check of primary key > + * modification. > + * @retrun Tuple containing result of upsert application; > + * NULL in case OOM. 6. retrun -> return. 7. I guess you are among the ones who voted for 80 symbol comments - I suggest you to use it. Since this is our new code style now. > + */ > +static struct tuple * > +vy_apply_upsert_on_terminal_stmt(struct tuple *upsert, struct tuple *stmt, > + struct key_def *cmp_def, bool suppress_error) > +{ > + assert(vy_stmt_type(upsert) == IPROTO_UPSERT); > + assert(stmt == NULL || vy_stmt_type(stmt) != IPROTO_UPSERT); > + > + uint32_t mp_size; > + const char *new_ops = vy_stmt_upsert_ops(upsert, &mp_size); > + /* Msgpack containing result of upserts application. */ > + const char *result_mp; > + if (vy_stmt_is_void(stmt)) 8. This {is_void} helper is used 2 times inside one funtion on the same value. Seems like you could simply inline it, remeber result into a variable {bool is_void;} and use it instead. > + result_mp = vy_upsert_data_range(upsert, &mp_size); > + else > + result_mp = tuple_data_range(stmt, &mp_size); > + const char *result_mp_end = result_mp + mp_size; > + /* > + * xrow_upsert_execute() allocates result using region, > + * so save starting point to release it later. > + */ > + struct region *region = &fiber()->gc; > + size_t region_svp = region_used(region); > + uint64_t column_mask = COLUMN_MASK_FULL; > + struct tuple_format *format = tuple_format(upsert); > + > + uint32_t ups_cnt = mp_decode_array(&new_ops); > + const char *ups_ops = new_ops; > + /* > + * In case upsert folds into insert, we must skip first > + * update operations. > + */ > + if (vy_stmt_is_void(stmt)) { > + ups_cnt--; > + mp_next(&ups_ops); > + } > + for (uint32_t i = 0; i < ups_cnt; ++i) { > + assert(mp_typeof(*ups_ops) == MP_ARRAY); > + const char *ups_ops_end = ups_ops; > + mp_next(&ups_ops_end); > + const char *exec_res = result_mp; > + exec_res = xrow_upsert_execute(ups_ops, ups_ops_end, result_mp, > + result_mp_end, format, &mp_size, > + 0, suppress_error, &column_mask); > + if (exec_res == NULL) { > + if (! suppress_error) { 9. According to one another recent code style change, unary operators should not have a whitespace after them. > + assert(diag_last_error(diag_get()) != NULL); 10. Use {diag_is_empty}. Or better - save {diag_last_error(diag_get())} into {struct error *e} before the assertion, and use {assert(e != NULL);}. > + struct error *e = diag_last_error(diag_get()); > + /* Bail out immediately in case of OOM. */ > + if (e->type != &type_ClientError) { > + region_truncate(region, region_svp); > + return NULL; > + } > + diag_log(); > + } > + ups_ops = ups_ops_end; > + continue; > + } > + /* > + * If it turns out that resulting tuple modifies primary > + * key, than simply ignore this upsert. 11. than -> then. > + */ > + if (vy_apply_result_does_cross_pk(stmt, exec_res, > + exec_res + mp_size, cmp_def, > + column_mask)) { > + if (! suppress_error) { > + say_error("upsert operations %s are not applied"\ > + " due to primary key modification", > + mp_str(ups_ops)); > + } > + ups_ops = ups_ops_end; > + continue; > + } > + ups_ops = ups_ops_end; > + /* > + * In case statement exists its format must > + * satisfy space's format. Otherwise, upsert's > + * tuple is checked to fit format once it is > + * processed in vy_upsert(). > + */ > + if (stmt != NULL) { > + if (tuple_validate_raw(tuple_format(stmt), > + exec_res) != 0) { > + if (! suppress_error) > + diag_log(); > + continue; > + } > + } > + result_mp = exec_res; > + result_mp_end = exec_res + mp_size; > + } > + struct tuple *new_terminal_stmt = vy_stmt_new_replace(format, result_mp, > + result_mp_end); > + region_truncate(region, region_svp); > + if (new_terminal_stmt == NULL) > + return NULL; > + vy_stmt_set_lsn(new_terminal_stmt, vy_stmt_lsn(upsert)); > + return new_terminal_stmt; > +} > + > +static bool > +tuple_format_is_suitable_for_squash(struct tuple_format *format) > +{ > + struct tuple_field *field; > + json_tree_foreach_entry_preorder(field, &format->fields.root, > + struct tuple_field, token) { > + if (field->type == FIELD_TYPE_UNSIGNED) > + return false; 12. Bad indentation. Also this squash rule is not going to work because integer type also can overflow, both below INT64_MIN and above UINT64_MAX. Decimal types can overflow. Besides, decimal can fail when a non-decimal value does not fit the decimal type during conversion. For example, a huge double value. DBL_MAX is bigger than maximal value available in our decimal type. See xrow_update_arith_make() for all errors. Since squash is mostly about squashing +/-, looks like it won't work almost always, and becomes useless. P.S. In the end of the review I noticed that this prevents squashing not only of operations with unsigned fields. It will prevent squashing if the whole format has at least one unsigned. This makes current implementation of squash even more useless, because forbids to use the fastest field type, which is default when you create an index without specification of field type btw. > + } > + return true; > +} > + > +/** > + * Unpack upsert's update operations from msgpack array > + * into array of iovecs. > + */ > +static void > +upsert_ops_to_iovec(const char *ops, uint32_t ops_cnt, struct iovec *iov_arr) > +{ > + for (uint32_t i = 0; i < ops_cnt; ++i) { > + assert(mp_typeof(*ops) == MP_ARRAY); > + iov_arr[i].iov_base = (char *) ops; > + mp_next(&ops); > + iov_arr[i].iov_len = ops - (char *) iov_arr[i].iov_base; > + } > +} > + > struct tuple * > vy_apply_upsert(struct tuple *new_stmt, struct tuple *old_stmt, > struct key_def *cmp_def, bool suppress_error) > @@ -87,122 +248,74 @@ vy_apply_upsert(struct tuple *new_stmt, struct tuple *old_stmt, > assert(new_stmt != old_stmt); > assert(vy_stmt_type(new_stmt) == IPROTO_UPSERT); > > - if (old_stmt == NULL || vy_stmt_type(old_stmt) == IPROTO_DELETE) { > - /* > - * INSERT case: return new stmt. > - */ > - return vy_stmt_replace_from_upsert(new_stmt); > + struct tuple *result_stmt = NULL; > + if (old_stmt == NULL || vy_stmt_type(old_stmt) != IPROTO_UPSERT) { > + return vy_apply_upsert_on_terminal_stmt(new_stmt, old_stmt, > + cmp_def, suppress_error); > } > > - struct tuple_format *format = tuple_format(new_stmt); > - > + assert(vy_stmt_type(old_stmt) == IPROTO_UPSERT); 13. The assertion looks useless, since it is reverse of the {if} condition above, but up to you. > /* > - * Unpack UPSERT operation from the new stmt > + * Unpack UPSERT operation from the old and new stmts. > */ > + assert(old_stmt != NULL); 14. This is strage to check old_stmt->type in the previous assertion before you checked old_stmt != NULL. > uint32_t mp_size; > - const char *new_ops; > - new_ops = vy_stmt_upsert_ops(new_stmt, &mp_size); > - const char *new_ops_end = new_ops + mp_size; > + const char *old_ops = vy_stmt_upsert_ops(old_stmt, &mp_size); > + const char *old_ops_end = old_ops + mp_size; > + assert(old_ops_end > old_ops); > + const char *old_stmt_mp = vy_upsert_data_range(old_stmt, &mp_size); > + const char *old_stmt_mp_end = old_stmt_mp + mp_size; > + const char *new_ops = vy_stmt_upsert_ops(new_stmt, &mp_size); > > /* > - * Apply new operations to the old stmt > + * UPSERT + UPSERT case: squash arithmetic operations. > + * Note that we can process this only in case result > + * can't break format under no circumstances. Since > + * subtraction can lead to negative values, unsigned > + * field are considered to be inappropriate. > */ > - const char *result_mp; > - if (vy_stmt_type(old_stmt) == IPROTO_UPSERT) > - result_mp = vy_upsert_data_range(old_stmt, &mp_size); > - else > - result_mp = tuple_data_range(old_stmt, &mp_size); > - const char *result_mp_end = result_mp + mp_size; > - struct tuple *result_stmt = NULL; > + struct tuple_format *format = tuple_format(old_stmt); > struct region *region = &fiber()->gc; > size_t region_svp = region_used(region); > - uint8_t old_type = vy_stmt_type(old_stmt); > - uint64_t column_mask = COLUMN_MASK_FULL; > - result_mp = xrow_upsert_execute(new_ops, new_ops_end, result_mp, > - result_mp_end, format, &mp_size, > - 0, suppress_error, &column_mask); > - if (result_mp == NULL) { > - region_truncate(region, region_svp); > - return NULL; > + if (tuple_format_is_suitable_for_squash(format)) { > + const char *new_ops_end = new_ops + mp_size; > + if (vy_upsert_try_to_squash(format, old_stmt_mp, old_stmt_mp_end, > + old_ops, old_ops_end, new_ops, > + new_ops_end, &result_stmt) != 0) { > + /* OOM */ > + region_truncate(region, region_svp); > + return NULL; > + } 15. vy_upsert_try_to_squash() returns a result into result_stmt. But you ignore it. Basically, whatever it returns, you act like squash didn't happen and it never works now. You continue to work with 2 old operation set arrays. Also result_stmt leaks. What is also strange - I added {assert(false);} here and the tests passed. I thought we had quite a lot squash tests. Seems they are all for formats having unsigned field type. (Actualy the tests failed, but not here - on my machine vinyl tests fail in almost 100% runs somewhere with random errors, could be luajit problems on Mac maybe.) > } > - result_mp_end = result_mp + mp_size; > - if (old_type != IPROTO_UPSERT) { > - assert(old_type == IPROTO_INSERT || > - old_type == IPROTO_REPLACE); > - /* > - * UPDATE case: return the updated old stmt. > - */ > - result_stmt = vy_stmt_new_replace(format, result_mp, > - result_mp_end); > - region_truncate(region, region_svp); > - if (result_stmt == NULL) > - return NULL; /* OOM */ > - vy_stmt_set_lsn(result_stmt, vy_stmt_lsn(new_stmt)); > - goto check_key; > - } > - > - /* > - * Unpack UPSERT operation from the old stmt > - */ > - assert(old_stmt != NULL); > - const char *old_ops; > - old_ops = vy_stmt_upsert_ops(old_stmt, &mp_size); > - const char *old_ops_end = old_ops + mp_size; > - assert(old_ops_end > old_ops); > - > /* > - * UPSERT + UPSERT case: combine operations > + * Adding update operations. We keep order of update operations in > + * the array the same. It is vital since first set of operations > + * must be skipped in case upsert folds into insert. For instance: > + * old_ops = {{{op1}, {op2}}, {{op3}}} > + * new_ops = {{{op4}, {op5}}} > + * res_ops = {{{op1}, {op2}}, {{op3}}, {{op4}, {op5}}} > + * If upsert corresponding to old_ops becomes insert, then > + * {{op1}, {op2}} update operations are not applied. > */ > - assert(old_ops_end - old_ops > 0); > - if (vy_upsert_try_to_squash(format, result_mp, result_mp_end, > - old_ops, old_ops_end, new_ops, new_ops_end, > - &result_stmt) != 0) { > + uint32_t old_ops_cnt = mp_decode_array(&old_ops); > + uint32_t new_ops_cnt = mp_decode_array(&new_ops); > + size_t ops_size = sizeof(struct iovec) * (old_ops_cnt + new_ops_cnt); > + struct iovec *operations = region_alloc(region, ops_size); 16. region_alloc_array. 17. But you don't really need that. Nor upsert_ops_to_iovec() function. You could keep the old code almost as is, because for vy_stmt_new_with_ops() to work correctly, it is not necessary to have each operation set in a separate iovec. Anyway they are all copied as is without unpacking. You could have 1 iovec for the root MP_ARRAY, 1 iovec for the old operation sets, 1 iovec for the new operation sets. Having first iovec with root MP_ARRAY would allow to delete is_ops_encoded. > + if (operations == NULL) { > region_truncate(region, region_svp); > + diag_set(OutOfMemory, ops_size, "region_alloc", "operations"); > return NULL; > } > - if (result_stmt != NULL) { > - region_truncate(region, region_svp); > - vy_stmt_set_lsn(result_stmt, vy_stmt_lsn(new_stmt)); > - goto check_key; > - } > + upsert_ops_to_iovec(old_ops, old_ops_cnt, operations); > + upsert_ops_to_iovec(new_ops, new_ops_cnt, &operations[old_ops_cnt]); > > - /* Failed to squash, simply add one upsert to another */ > - int old_ops_cnt, new_ops_cnt; > - struct iovec operations[3]; > - > - old_ops_cnt = mp_decode_array(&old_ops); > - operations[1].iov_base = (void *)old_ops; > - operations[1].iov_len = old_ops_end - old_ops; > - > - new_ops_cnt = mp_decode_array(&new_ops); > - operations[2].iov_base = (void *)new_ops; > - operations[2].iov_len = new_ops_end - new_ops; > - > - char ops_buf[16]; > - char *header = mp_encode_array(ops_buf, old_ops_cnt + new_ops_cnt); > - operations[0].iov_base = (void *)ops_buf; > - operations[0].iov_len = header - ops_buf; > - > - result_stmt = vy_stmt_new_upsert(format, result_mp, result_mp_end, > - operations, 3); > + result_stmt = vy_stmt_new_upsert(format, old_stmt_mp, old_stmt_mp_end, > + operations, old_ops_cnt + new_ops_cnt, > + false); > region_truncate(region, region_svp); > if (result_stmt == NULL) > return NULL; > vy_stmt_set_lsn(result_stmt, vy_stmt_lsn(new_stmt)); > > -check_key: > - /* > - * Check that key hasn't been changed after applying operations. > - */ > - if (!key_update_can_be_skipped(cmp_def->column_mask, column_mask) && > - vy_stmt_compare(old_stmt, HINT_NONE, result_stmt, > - HINT_NONE, cmp_def) != 0) { > - /* > - * Key has been changed: ignore this UPSERT and > - * @retval the old stmt. > - */ > - tuple_unref(result_stmt); > - result_stmt = vy_stmt_dup(old_stmt); > - } > return result_stmt; > diff --git a/test/vinyl/upsert.result b/test/vinyl/upsert.result > index 3a7f6629d..a20db2ad2 100644 > --- a/test/vinyl/upsert.result > +++ b/test/vinyl/upsert.result > @@ -899,3 +899,292 @@ s:select() > s:drop() > --- > ... > +-- gh-5107: don't squash upsert operations into one array. > +-- > +-- Test upsert execution/squash referring to fields in reversed > +-- order (via negative indexing). > +-- > +s = box.schema.create_space('test', {engine = 'vinyl'}) > +--- > +... > +pk = s:create_index('pk') > +--- > +... > +s:insert({1, 1, 1}) > +--- > +- [1, 1, 1] > +... > +box.snapshot() > +--- > +- ok > +... > +s:upsert({1}, {{'=', 3, 100}}) > +--- > +... > +s:upsert({1}, {{'=', -1, 200}}) > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +s:select() -- {1, 1, 200} > +--- > +- - [1, 1, 200] > +... > +s:delete({1}) > +--- > +... > +s:insert({1, 1, 1}) > +--- > +- [1, 1, 1] > +... > +box.snapshot() > +--- > +- ok > +... > +s:upsert({1}, {{'=', -3, 100}}) > +--- > +... > +s:upsert({1}, {{'=', -1, 200}}) > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +-- Two upserts are NOT squashed into one, so only one > +-- (first one) is skipped, meanwhile second one is applied. > +-- > +s:select() -- {1, 1, 1} > +--- > +- - [1, 1, 200] > +... > +s:delete({1}) > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +s:upsert({1, 1}, {{'=', -2, 300}}) -- {1, 1} > +--- > +... > +s:upsert({1}, {{'+', -1, 100}}) -- {1, 101} > +--- > +... > +s:upsert({1}, {{'-', 2, 100}}) -- {1, 1} > +--- > +... > +s:upsert({1}, {{'+', -1, 200}}) -- {1, 201} > +--- > +... > +s:upsert({1}, {{'-', 2, 200}}) -- {1, 1} > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +s:select() -- {1, 1} > +--- > +- - [1, 1] > +... > +s:delete({1}) > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +s:upsert({1, 1, 1}, {{'!', -1, 300}}) -- {1, 1, 1} > +--- > +... > +s:upsert({1}, {{'+', -2, 100}}) -- {1, 101, 1} > +--- > +... > +s:upsert({1}, {{'=', -1, 100}}) -- {1, 101, 100} > +--- > +... > +s:upsert({1}, {{'+', -1, 200}}) -- {1, 101, 300} > +--- > +... > +s:upsert({1}, {{'-', -2, 100}}) -- {1, 1, 300} > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +s:select() > +--- > +- - [1, 1, 300] > +... > +s:drop() > +--- > +... > +-- Upsert operations which break space format are not applied. 18. You need to put references to the relevant issues in the tests, using -- -- gh-NNNN: description. -- format. > +-- > +s = box.schema.space.create('test', { engine = 'vinyl', field_count = 2 }) > +--- > +... > +pk = s:create_index('pk') > +--- > +... > +s:replace{1, 1} > +--- > +- [1, 1] > +... > +-- Error is logged, upsert is not applied. > +-- > +s:upsert({1, 1}, {{'=', 3, 5}}) > +--- > +... > +-- During read the incorrect upsert is ignored. > +-- > +s:select{} > +--- > +- - [1, 1] > +... > +-- Try to set incorrect field_count in a transaction. > +-- > +box.begin() > +--- > +... > +s:replace{2, 2} > +--- > +- [2, 2] > +... > +s:upsert({2, 2}, {{'=', 3, 2}}) > +--- > +... > +s:select{} > +--- > +- - [1, 1] > + - [2, 2] > +... > +box.commit() > +--- > +... > +s:select{} > +--- > +- - [1, 1] > + - [2, 2] > +... > +-- Read incorrect upsert from a run: it should be ignored. > +-- > +box.snapshot() > +--- > +- ok > +... > +s:select{} > +--- > +- - [1, 1] > + - [2, 2] > +... > +s:upsert({2, 2}, {{'=', 3, 20}}) > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +s:select{} > +--- > +- - [1, 1] > + - [2, 2] > +... > +-- Execute replace/delete after invalid upsert. > +-- > +box.snapshot() > +--- > +- ok > +... > +s:upsert({2, 2}, {{'=', 3, 30}}) > +--- > +... > +s:replace{2, 3} > +--- > +- [2, 3] > +... > +s:select{} > +--- > +- - [1, 1] > + - [2, 3] > +... > +s:upsert({1, 1}, {{'=', 3, 30}}) > +--- > +... > +s:delete{1} > +--- > +... > +s:select{} > +--- > +- - [2, 3] > +... > +-- Invalid upsert in a sequence of upserts is skipped meanwhile > +-- the rest are applied. > +-- > +box.snapshot() > +--- > +- ok > +... > +s:upsert({2, 2}, {{'+', 2, 5}}) > +--- > +... > +s:upsert({2, 2}, {{'=', 3, 40}}) > +--- > +... > +s:upsert({2, 2}, {{'+', 2, 5}}) > +--- > +... > +s:select{} > +--- > +- - [2, 13] > +... > +box.snapshot() > +--- > +- ok > +... > +s:select{} > +--- > +- - [2, 13] > +... > +s:drop() > +--- > +... > +-- Make sure upserts satisfy associativity rule. > +-- > +s = box.schema.space.create('test', {engine='vinyl'}) > +--- > +... > +i = s:create_index('pk', {parts={2, 'uint'}}) > +--- > +... > +s:replace{1, 2, 3, 'default'} > +--- > +- [1, 2, 3, 'default'] > +... > +box.snapshot() > +--- > +- ok > +... > +s:upsert({2, 2, 2}, {{'=', 4, 'upserted'}}) > +--- > +... > +-- Upsert will fail and thus ignored. > +-- > +s:upsert({2, 2, 2}, {{'#', 1, 1}, {'!', 3, 1}}) > +--- > +... > +box.snapshot() > +--- > +- ok > +... > +s:select{} > +--- > +- - [1, 2, 3, 'upserted'] > +... > +s:drop() > +--- > +... 19. All tests work with unsigned fields. So squashing is not tested here. From korablev at tarantool.org Mon Aug 3 11:42:35 2020 From: korablev at tarantool.org (Nikita Pettik) Date: Mon, 3 Aug 2020 08:42:35 +0000 Subject: [Tarantool-patches] [PATCH] tuple: drop extra restrictions for multikey index In-Reply-To: <1596201198.417099977@f747.i.mail.ru> References: <20200724200633.12122-1-i.kosarev@tarantool.org> <20200730142620.GA25238@tarantool.org> <1596201198.417099977@f747.i.mail.ru> Message-ID: <20200803084235.GA8230@tarantool.org> On 31 Jul 16:13, Ilya Kosarev wrote: > > Hi! > ? > Thanks for your review. > ? > See my answers below. > ? > I will send v2 of the patch but i think 2 questions above?should be clarified. > ? > >???????, 30 ???? 2020, 17:26 +03:00 ?? Nikita Pettik : > >? > >On 24 Jul 23:06, Ilya Kosarev wrote: > > > >I *strongly dislike* that the patch is in fact should be 10 lines > >but instead we have 250 diff lines. Please elaborate on that > >(comments above are mostly about that). > Well, this patch reverts the majority of changes from > 4cf94ef8cb90b84ea71f313cff3e016f85894fd5 (tuple: make fields nullable > by default except array/map). That is where the size comes from. Then you are able simply to revert that patch and introduce new one. But I'd provide only required fixes. > >> diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c > >> index 8452ab430..875592026 100644 > >> --- a/src/box/memtx_space.c > >> +++ b/src/box/memtx_space.c > >> @@ -589,9 +589,7 @@ memtx_space_ephemeral_rowid_next(struct space *space, uint64_t *rowid) > >> static int > >> memtx_space_check_index_def(struct space *space, struct index_def *index_def) > >> { > >> - struct key_def *key_def = index_def->key_def; > >> - > >> - if (key_def->is_nullable) { > >> + if (index_def->key_def->is_nullable) { > > > >Again I see this redundant diff. Please drop it, it doesn't fix > >refactor/fix anything but makes diff bigger and complicates git > >history. > The problem is that this is the reversion of that diff. Well, I guess > as far as that patch was pushed to master we can leave that change > there and revert only the meaningful part? Yes. From sergos at tarantool.org Mon Aug 3 23:53:28 2020 From: sergos at tarantool.org (sergos at tarantool.org) Date: Mon, 3 Aug 2020 23:53:28 +0300 Subject: [Tarantool-patches] [RFC] Raft over qsync Message-ID: <4E8CD3F1-803C-494A-AC81-B26968280E04@tarantool.org> * **Status**: In progress * **Start date**: 27-07-2020 * **Authors**: Sergey Ostanevich @sergos \, Vladislav Shpilevoy @Gerold103 \, Cyrill Gorcunov @cyrillos \ * **Issues**: https://github.com/tarantool/tarantool/issues/5202 ## Summary The #4842 is introduced a quorum based replication (qsync) in Tarantool environment. To augment the synchronous replication with automated leader election and failover, we need to make a choice on how to implement one of algorithms available. Our preference is Raft since it is has good comprehensibility. I would refer to the https://raft.github.io/raft.pdf further in the document. The biggest problem is to link together the master-master nature of log replication in Tarantool with the strong leader concept in Raft. ## Background and motivation Customers requested synchronous replication with automated failover as part of Tarantool development. These features also fits well with Tarantool future we envision in MRG. ## Detailed design Qsync implementation is performed in a way that allows users to run Tarantool-based solution without any changes to their code base. Literally if you have a database set up, it will continue running after upgrade the same way as it was prior to 2.5 version. You can start employing the synchronous replication by introduction of specific spaces in your schema and after that qsync with come to play. There were no changes to the protocol, so you can mix 2.5 instances with previous in both ways - replicate from new to old either vice-versa - until you introduce the first synchronous space. The machinery under the hood oblige all instances to follow a new process of transaction: if transaction touches a synchronous space then it will require a special command from the WAL - confirm. Since the obligation is an incremental requirement we can keep the backward compatibility and in both ways. We expect to elaborate similar approach to the Raft-based failover machinery. Which means one can use the qsync replication without the Raft enabled, being able to elaborate its own failover mechanism. Although, if Raft is enabled then all instances in the cluster are obliged to follow the rules implied by the Raft, such as ignore log entries from a leader with stale term number. ### Leader Election We expect the leader election algorithm can be reused from the Raft implementation [2] with additional mapping to the Tarantool replication mechanism. The replication mechanism of Tarantool with qsync provides the following features, that required by Raft algorithm: * The new entry is delivered to the majority of the replicas and only after this it is committed, the fact of commit is delivered to client * The log reflects the fact of an entry is committed by a special entry 'Commit' as part of qsync machinery * Log consistency comes from the append-only nature of the log and the vclock check-up during the log append * Log inconsistencies are handled during replication (see log replication below) Raft implementation will hold leader's term and it's vote for the current term in its local structures, while issuing WAL entries to reflect changes in the status will ensure the persistence of the Raft state. After a fail node will read the WAL entries and apply them to its runtime state. These entries should go into WAL under the ID number 0, so that they will not be propagated. To propagate the results of election there should be an entry in WAL with dedicated info: raft_term and leader_id. An elected leader should issue such an entry upon its election as a first entry. ### Log Replication The qsync RFC explains how we enforce the log replication in a way it is described in clause 5.3 of the [1]: committed entry always has a commit message in the xlog. Key difference here is that log entry index comprises of two parts: the LSN and the served ID. The follower's log consistency will be achieved during a) leader election, when follower will only vote for a candidate who has VCLOCK components greater or equal to follower's and b) during the join to a new leader, when follower will have an option to drop it's waiting queue (named limbo in qsync implementation), either perform a full rejoin. The latter is painful, still is the only way to follow the current representation of xlog that contains no replay info. There is a requirement in 5.1 of [1]: > If a server receives a request with a stale term number, it rejects the > request. that requires to introduce a machinery that will inspect every DML message from IPROTO to check if it satisfies the requirement. To introduce this machinery there should be an additional info put into the replicated DML map under IPROTO_RAFT_TERM: a term of the leader taken from the Raft runtime implementation. This info should be added to the DML - and checked - only in case the cluster is configured to use the Raft machinery. As a result an instance that is joining a cluster with the Raft protocol enabled has to enable the protocol either, otherwise it should be disconnected. ## Rationale and alternatives C implementation of Raft using binary protocol has an alternative of implementation using Lua, for example [3]. Although, the performance of the latter can have a significant impact in part of log replication enforcement. ## References * [1] https://raft.github.io/raft.pdf * [2] https://github.com/willemt/raft * [3] https://github.com/igorcoding/tarantool-raft From kostja.osipov at gmail.com Mon Aug 3 23:56:28 2020 From: kostja.osipov at gmail.com (Konstantin Osipov) Date: Mon, 3 Aug 2020 23:56:28 +0300 Subject: [Tarantool-patches] [RFC] Raft over qsync In-Reply-To: <4E8CD3F1-803C-494A-AC81-B26968280E04@tarantool.org> References: <4E8CD3F1-803C-494A-AC81-B26968280E04@tarantool.org> Message-ID: <20200803205628.GA387134@atlas> * sergos at tarantool.org [20/08/03 23:54]: willem is a very naive implementation, I'd recommend looking at etcd/raft instead. > * **Status**: In progress > * **Start date**: 27-07-2020 > * **Authors**: Sergey Ostanevich @sergos \, Vladislav Shpilevoy @Gerold103 \, Cyrill Gorcunov @cyrillos \ > * **Issues**: https://github.com/tarantool/tarantool/issues/5202 > > ## Summary > > The #4842 is introduced a quorum based replication (qsync) in Tarantool > environment. To augment the synchronous replication with automated leader > election and failover, we need to make a choice on how to implement one of > algorithms available. Our preference is Raft since it is has good > comprehensibility. I would refer to the https://raft.github.io/raft.pdf further > in the document. > > The biggest problem is to link together the master-master nature of log > replication in Tarantool with the strong leader concept in Raft. > > ## Background and motivation > > Customers requested synchronous replication with automated failover as part of > Tarantool development. These features also fits well with Tarantool future we > envision in MRG. > > ## Detailed design > > Qsync implementation is performed in a way that allows users to run > Tarantool-based solution without any changes to their code base. Literally if > you have a database set up, it will continue running after upgrade the same way > as it was prior to 2.5 version. You can start employing the synchronous > replication by introduction of specific spaces in your schema and after that > qsync with come to play. There were no changes to the protocol, so you can mix > 2.5 instances with previous in both ways - replicate from new to old either > vice-versa - until you introduce the first synchronous space. > > The machinery under the hood oblige all instances to follow a new process of > transaction: if transaction touches a synchronous space then it will require a > special command from the WAL - confirm. Since the obligation is an incremental > requirement we can keep the backward compatibility and in both ways. > > We expect to elaborate similar approach to the Raft-based failover machinery. > Which means one can use the qsync replication without the Raft enabled, being > able to elaborate its own failover mechanism. Although, if Raft is enabled then > all instances in the cluster are obliged to follow the rules implied by the > Raft, such as ignore log entries from a leader with stale term number. > > ### Leader Election > > We expect the leader election algorithm can be reused from the Raft > implementation [2] with additional mapping to the Tarantool replication > mechanism. > > The replication mechanism of Tarantool with qsync provides the following > features, that required by Raft algorithm: > * The new entry is delivered to the majority of the replicas and only after this > it is committed, the fact of commit is delivered to client > * The log reflects the fact of an entry is committed by a special entry 'Commit' > as part of qsync machinery > * Log consistency comes from the append-only nature of the log and the vclock > check-up during the log append > * Log inconsistencies are handled during replication (see log replication below) > > Raft implementation will hold leader's term and it's vote for the current term > in its local structures, while issuing WAL entries to reflect changes in the > status will ensure the persistence of the Raft state. After a fail node will > read the WAL entries and apply them to its runtime state. These entries should > go into WAL under the ID number 0, so that they will not be propagated. > > To propagate the results of election there should be an entry in WAL with > dedicated info: raft_term and leader_id. An elected leader should issue such an > entry upon its election as a first entry. > > > ### Log Replication > > The qsync RFC explains how we enforce the log replication in a way it is > described in clause 5.3 of the [1]: committed entry always has a commit message > in the xlog. Key difference here is that log entry index comprises of two parts: > the LSN and the served ID. The follower's log consistency will be achieved > during a) leader election, when follower will only vote for a candidate who has > VCLOCK components greater or equal to follower's and b) during the join to a new > leader, when follower will have an option to drop it's waiting queue (named > limbo in qsync implementation), either perform a full rejoin. > The latter is painful, still is the only way to follow the current > representation of xlog that contains no replay info. > > There is a requirement in 5.1 of [1]: > > > If a server receives a request with a stale term number, it rejects the > > request. > > that requires to introduce a machinery that will inspect every DML message from > IPROTO to check if it satisfies the requirement. To introduce this machinery > there should be an additional info put into the replicated DML map under > IPROTO_RAFT_TERM: a term of the leader taken from the Raft runtime > implementation. This info should be added to the DML - and checked - only in > case the cluster is configured to use the Raft machinery. > > As a result an instance that is joining a cluster with the Raft protocol enabled > has to enable the protocol either, otherwise it should be disconnected. > > ## Rationale and alternatives > > C implementation of Raft using binary protocol has an alternative of > implementation using Lua, for example [3]. Although, the performance of the > latter can have a significant impact in part of log replication enforcement. > > ## References > > * [1] https://raft.github.io/raft.pdf > * [2] https://github.com/willemt/raft > * [3] https://github.com/igorcoding/tarantool-raft -- Konstantin Osipov, Moscow, Russia From kyukhin at tarantool.org Tue Aug 4 10:28:41 2020 From: kyukhin at tarantool.org (Kirill Yukhin) Date: Tue, 4 Aug 2020 10:28:41 +0300 Subject: [Tarantool-patches] [PATCH v2 0/2] fiber: backport for stack madvise/mprotect errors handling In-Reply-To: References: Message-ID: <20200804072841.kehwy2prt2ymsjvz@tarantool.org> Hello, On 29 ??? 22:44, Ilya Kosarev wrote: > This patchset is a backport for > c67522973b32fae955835109d4f86eada8f67ae5 and > 8d53fadc0cb15a45ea0cd461d2d5243be51b37e0 patches on stack > madvise/mprotect errors handling. > > Branch: https://github.com/tarantool/tarantool/tree/i.kosarev/gh-5211-diag-at-fiber-mprotect-and-madvise > Issue: https://github.com/tarantool/tarantool/issues/5211 LGTM. I've checked your patchset into 1.10. -- Regards, Kirill Yukhin From i.kosarev at tarantool.org Tue Aug 4 12:55:55 2020 From: i.kosarev at tarantool.org (=?UTF-8?B?SWx5YSBLb3NhcmV2?=) Date: Tue, 04 Aug 2020 12:55:55 +0300 Subject: [Tarantool-patches] =?utf-8?q?=5BPATCH_v2_1/3=5D_alter=3A_use_goo?= =?utf-8?q?d_c++_style?= In-Reply-To: <1594649887-15890-2-git-send-email-alyapunov@tarantool.org> References: <1594649887-15890-1-git-send-email-alyapunov@tarantool.org> <1594649887-15890-2-git-send-email-alyapunov@tarantool.org> Message-ID: <1596534955.779570951@f731.i.mail.ru> Hi! Yes, this?patch seems to be nice. ? See 1 comment below. >???????????, 13 ???? 2020, 17:18 +03:00 ?? Aleksandr Lyapunov : >? >We should use noexcept where it's appropriate. >1.It informs a developer he could not care about exception safety >2.It instructs a compiler to check throws if possible > >We should override for compile-time check of misprints in names >of methods and types of arguments. > >Part of #5153 >--- >?src/box/alter.cc | 180 +++++++++++++++++++++++++++---------------------------- >?1 file changed, 90 insertions(+), 90 deletions(-) > >diff --git a/src/box/alter.cc b/src/box/alter.cc >index 249cee2..15a51d0 100644 >--- a/src/box/alter.cc >+++ b/src/box/alter.cc >@@ -765,7 +765,7 @@ struct alter_space; >? >?class AlterSpaceOp { >?public: >- AlterSpaceOp(struct alter_space *alter); >+ explicit AlterSpaceOp(struct alter_space *alter); >? >? /** Link in alter_space::ops. */ >? struct rlist link; >@@ -774,7 +774,7 @@ public: >? * the space definition and/or key list that will be used >? * for creating the new space. Must not yield or fail. >? */ >- virtual void alter_def(struct alter_space * /* alter */) {} >+ virtual void alter_def(struct alter_space * /* alter */) noexcept {} >? /** >? * Called after creating a new space. Used for performing >? * long-lasting operations, such as index rebuild or format >@@ -788,21 +788,21 @@ public: >? * state to the new space (e.g. move unchanged indexes). >? * Must not yield or fail. >? */ >- virtual void alter(struct alter_space * /* alter */) {} >+ virtual void alter(struct alter_space * /* alter */) noexcept {} >? /** >? * Called after the change has been successfully written >? * to WAL. Must not fail. >? */ >? virtual void commit(struct alter_space * /* alter */, >- int64_t /* signature */) {} >+ int64_t /* signature */) noexcept {} >? /** >? * Called in case a WAL error occurred. It is supposed to undo >? * the effect of AlterSpaceOp::prepare and AlterSpaceOp::alter. >? * Must not fail. >? */ >- virtual void rollback(struct alter_space * /* alter */) {} >+ virtual void rollback(struct alter_space * /* alter */) noexcept {} >? >- virtual ~AlterSpaceOp() {} >+ virtual ~AlterSpaceOp() noexcept {} >? >? void *operator new(size_t size) >? { >@@ -984,7 +984,7 @@ struct mh_i32_t *AlterSpaceLock::registry; >??* Replace the old space with a new one in the space cache. >??*/ >?static int >-alter_space_commit(struct trigger *trigger, void *event) >+alter_space_commit(struct trigger *trigger, void *event) noexcept >?{ >? struct txn *txn = (struct txn *) event; >? struct alter_space *alter = (struct alter_space *) trigger->data; >@@ -1002,12 +1002,8 @@ alter_space_commit(struct trigger *trigger, void *event) >? * indexes into their new places. >? */ >? class AlterSpaceOp *op; >- try { >- rlist_foreach_entry(op, &alter->ops, link) >- op->commit(alter, signature); >- } catch (Exception *e) { >- return -1; >- } >+ rlist_foreach_entry(op, &alter->ops, link) >+ op->commit(alter, signature); I guess this change should be in ?alter: do not catch exceptions when it's not necessary? commit. >? >? alter->new_space = NULL; /* for alter_space_delete(). */ >? /* >@@ -1029,7 +1025,7 @@ alter_space_commit(struct trigger *trigger, void *event) >??* alter_space_commit() failure (unlikely) >??*/ >?static int >-alter_space_rollback(struct trigger *trigger, void * /* event */) >+alter_space_rollback(struct trigger *trigger, void * /* event */) noexcept >?{ >? struct alter_space *alter = (struct alter_space *) trigger->data; >? /* Rollback alter ops */ >@@ -1188,9 +1184,9 @@ alter_space_do(struct txn_stmt *stmt, struct alter_space *alter) >?class CheckSpaceFormat: public AlterSpaceOp >?{ >?public: >- CheckSpaceFormat(struct alter_space *alter) >+ explicit CheckSpaceFormat(struct alter_space *alter) >? :AlterSpaceOp(alter) {} >- virtual void prepare(struct alter_space *alter); >+ void prepare(struct alter_space *alter) override; >?}; >? >?void >@@ -1222,15 +1218,15 @@ public: >? * names into an old dictionary and deletes new one. >? */ >? struct tuple_dictionary *new_dict; >- virtual void alter_def(struct alter_space *alter); >- virtual void alter(struct alter_space *alter); >- virtual void rollback(struct alter_space *alter); >- virtual ~ModifySpace(); >+ void alter_def(struct alter_space *alter) noexcept override; >+ void alter(struct alter_space *alter) noexcept override; >+ void rollback(struct alter_space *alter) noexcept override; >+ ~ModifySpace() noexcept override; >?}; >? >?/** Amend the definition of the new space. */ >?void >-ModifySpace::alter_def(struct alter_space *alter) >+ModifySpace::alter_def(struct alter_space *alter) noexcept >?{ >? /* >? * Use the old dictionary for the new space, because >@@ -1249,7 +1245,7 @@ ModifySpace::alter_def(struct alter_space *alter) >?} >? >?void >-ModifySpace::alter(struct alter_space *alter) >+ModifySpace::alter(struct alter_space *alter) noexcept >?{ >? /* >? * Move new names into an old dictionary, which already is >@@ -1260,12 +1256,12 @@ ModifySpace::alter(struct alter_space *alter) >?} >? >?void >-ModifySpace::rollback(struct alter_space *alter) >+ModifySpace::rollback(struct alter_space *alter) noexcept >?{ >? tuple_dictionary_swap(alter->new_space->def->dict, new_dict); >?} >? >-ModifySpace::~ModifySpace() >+ModifySpace::~ModifySpace() noexcept >?{ >? if (new_dict != NULL) >? tuple_dictionary_unref(new_dict); >@@ -1281,9 +1277,9 @@ public: >? DropIndex(struct alter_space *alter, struct index *index) >? :AlterSpaceOp(alter), old_index(index) {} >? struct index *old_index; >- virtual void alter_def(struct alter_space *alter); >- virtual void prepare(struct alter_space *alter); >- virtual void commit(struct alter_space *alter, int64_t lsn); >+ void alter_def(struct alter_space *alter) noexcept override; >+ void prepare(struct alter_space *alter) override; >+ void commit(struct alter_space *alter, int64_t lsn) noexcept override; >?}; >? >?/* >@@ -1291,7 +1287,7 @@ public: >??* the new index from it. >??*/ >?void >-DropIndex::alter_def(struct alter_space * /* alter */) >+DropIndex::alter_def(struct alter_space * /* alter */) noexcept >?{ >? rlist_del_entry(old_index->def, link); >?} >@@ -1305,7 +1301,7 @@ DropIndex::prepare(struct alter_space *alter) >?} >? >?void >-DropIndex::commit(struct alter_space *alter, int64_t signature) >+DropIndex::commit(struct alter_space *alter, int64_t signature) noexcept >?{ >? (void)alter; >? index_commit_drop(old_index, signature); >@@ -1323,18 +1319,18 @@ public: >? :AlterSpaceOp(alter), iid(iid_arg) {} >? /** id of the index on the move. */ >? uint32_t iid; >- virtual void alter(struct alter_space *alter); >- virtual void rollback(struct alter_space *alter); >+ void alter(struct alter_space *alter) noexcept override; >+ void rollback(struct alter_space *alter) noexcept override; >?}; >? >?void >-MoveIndex::alter(struct alter_space *alter) >+MoveIndex::alter(struct alter_space *alter) noexcept >?{ >? space_swap_index(alter->old_space, alter->new_space, iid, iid); >?} >? >?void >-MoveIndex::rollback(struct alter_space *alter) >+MoveIndex::rollback(struct alter_space *alter) noexcept >?{ >? space_swap_index(alter->old_space, alter->new_space, iid, iid); >?} >@@ -1365,23 +1361,23 @@ public: >? struct index *old_index; >? struct index *new_index; >? struct index_def *new_index_def; >- virtual void alter_def(struct alter_space *alter); >- virtual void alter(struct alter_space *alter); >- virtual void commit(struct alter_space *alter, int64_t lsn); >- virtual void rollback(struct alter_space *alter); >- virtual ~ModifyIndex(); >+ void alter_def(struct alter_space *alter) noexcept override; >+ void alter(struct alter_space *alter) noexcept override; >+ void commit(struct alter_space *alter, int64_t lsn) noexcept override; >+ void rollback(struct alter_space *alter) noexcept override; >+ ~ModifyIndex() noexcept override; >?}; >? >?/** Update the definition of the new space */ >?void >-ModifyIndex::alter_def(struct alter_space *alter) >+ModifyIndex::alter_def(struct alter_space *alter) noexcept >?{ >? rlist_del_entry(old_index->def, link); >? index_def_list_add(&alter->key_list, new_index_def); >?} >? >?void >-ModifyIndex::alter(struct alter_space *alter) >+ModifyIndex::alter(struct alter_space *alter) noexcept >?{ >? new_index = space_index(alter->new_space, new_index_def->iid); >? assert(old_index->def->iid == new_index->def->iid); >@@ -1397,14 +1393,14 @@ ModifyIndex::alter(struct alter_space *alter) >?} >? >?void >-ModifyIndex::commit(struct alter_space *alter, int64_t signature) >+ModifyIndex::commit(struct alter_space *alter, int64_t signature) noexcept >?{ >? (void)alter; >? index_commit_modify(new_index, signature); >?} >? >?void >-ModifyIndex::rollback(struct alter_space *alter) >+ModifyIndex::rollback(struct alter_space *alter) noexcept >?{ >? /* >? * Restore indexes. >@@ -1416,7 +1412,7 @@ ModifyIndex::rollback(struct alter_space *alter) >? index_update_def(old_index); >?} >? >-ModifyIndex::~ModifyIndex() >+ModifyIndex::~ModifyIndex() noexcept >?{ >? index_def_delete(new_index_def); >?} >@@ -1432,15 +1428,15 @@ public: >? CreateIndex(struct alter_space *alter, struct index_def *def) >? :AlterSpaceOp(alter), new_index(NULL), new_index_def(def) >? {} >- virtual void alter_def(struct alter_space *alter); >- virtual void prepare(struct alter_space *alter); >- virtual void commit(struct alter_space *alter, int64_t lsn); >- virtual ~CreateIndex(); >+ void alter_def(struct alter_space *alter) noexcept override; >+ void prepare(struct alter_space *alter) override; >+ void commit(struct alter_space *alter, int64_t lsn) noexcept override; >+ ~CreateIndex() noexcept override; >?}; >? >?/** Add definition of the new key to the new space def. */ >?void >-CreateIndex::alter_def(struct alter_space *alter) >+CreateIndex::alter_def(struct alter_space *alter) noexcept >?{ >? index_def_list_add(&alter->key_list, new_index_def); >?} >@@ -1480,7 +1476,7 @@ CreateIndex::prepare(struct alter_space *alter) >?} >? >?void >-CreateIndex::commit(struct alter_space *alter, int64_t signature) >+CreateIndex::commit(struct alter_space *alter, int64_t signature) noexcept >?{ >? (void) alter; >? assert(new_index != NULL); >@@ -1488,7 +1484,7 @@ CreateIndex::commit(struct alter_space *alter, int64_t signature) >? new_index = NULL; >?} >? >-CreateIndex::~CreateIndex() >+CreateIndex::~CreateIndex() noexcept >?{ >? if (new_index != NULL) >? index_abort_create(new_index); >@@ -1521,15 +1517,15 @@ public: >? struct index_def *new_index_def; >? /** Old index index_def. */ >? struct index_def *old_index_def; >- virtual void alter_def(struct alter_space *alter); >- virtual void prepare(struct alter_space *alter); >- virtual void commit(struct alter_space *alter, int64_t signature); >- virtual ~RebuildIndex(); >+ void alter_def(struct alter_space *alter) noexcept override; >+ void prepare(struct alter_space *alter) override; >+ void commit(struct alter_space *alter, int64_t signature) noexcept override; >+ ~RebuildIndex() noexcept override; >?}; >? >?/** Add definition of the new key to the new space def. */ >?void >-RebuildIndex::alter_def(struct alter_space *alter) >+RebuildIndex::alter_def(struct alter_space *alter) noexcept >?{ >? rlist_del_entry(old_index_def, link); >? index_def_list_add(&alter->key_list, new_index_def); >@@ -1545,7 +1541,7 @@ RebuildIndex::prepare(struct alter_space *alter) >?} >? >?void >-RebuildIndex::commit(struct alter_space *alter, int64_t signature) >+RebuildIndex::commit(struct alter_space *alter, int64_t signature) noexcept >?{ >? struct index *old_index = space_index(alter->old_space, >? old_index_def->iid); >@@ -1556,7 +1552,7 @@ RebuildIndex::commit(struct alter_space *alter, int64_t signature) >? new_index = NULL; >?} >? >-RebuildIndex::~RebuildIndex() >+RebuildIndex::~RebuildIndex() noexcept >?{ >? if (new_index != NULL) >? index_abort_create(new_index); >@@ -1600,9 +1596,10 @@ public: >? TruncateIndex(struct alter_space *alter, uint32_t iid) >? : AlterSpaceOp(alter), iid(iid), >? old_index(NULL), new_index(NULL) {} >- virtual void prepare(struct alter_space *alter); >- virtual void commit(struct alter_space *alter, int64_t signature); >- virtual ~TruncateIndex(); >+ void prepare(struct alter_space *alter) override; >+ void commit(struct alter_space *alter, >+ int64_t signature) noexcept override; >+ ~TruncateIndex() noexcept override; >?}; >? >?void >@@ -1632,7 +1629,7 @@ TruncateIndex::prepare(struct alter_space *alter) >?} >? >?void >-TruncateIndex::commit(struct alter_space *alter, int64_t signature) >+TruncateIndex::commit(struct alter_space *alter, int64_t signature) noexcept >?{ >? (void)alter; >? index_commit_drop(old_index, signature); >@@ -1640,7 +1637,7 @@ TruncateIndex::commit(struct alter_space *alter, int64_t signature) >? new_index = NULL; >?} >? >-TruncateIndex::~TruncateIndex() >+TruncateIndex::~TruncateIndex() noexcept >?{ >? if (new_index == NULL) >? return; >@@ -1657,11 +1654,11 @@ class UpdateSchemaVersion: public AlterSpaceOp >?public: >? UpdateSchemaVersion(struct alter_space * alter) >? :AlterSpaceOp(alter) {} >- virtual void alter(struct alter_space *alter); >+ void alter(struct alter_space *alter) noexcept override; >?}; >? >?void >-UpdateSchemaVersion::alter(struct alter_space *alter) >+UpdateSchemaVersion::alter(struct alter_space *alter) noexcept >?{ >?????(void)alter; >?????++schema_version; >@@ -1684,10 +1681,10 @@ public: >? RebuildCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter), >? ck_constraint(RLIST_HEAD_INITIALIZER(ck_constraint)) {} >? struct rlist ck_constraint; >- virtual void prepare(struct alter_space *alter); >- virtual void alter(struct alter_space *alter); >- virtual void rollback(struct alter_space *alter); >- virtual ~RebuildCkConstraints(); >+ void prepare(struct alter_space *alter) override; >+ void alter(struct alter_space *alter) noexcept override; >+ void rollback(struct alter_space *alter) noexcept override; >+ ~RebuildCkConstraints() noexcept override; >?}; >? >?void >@@ -1716,18 +1713,18 @@ RebuildCkConstraints::space_swap_ck_constraint(struct space *old_space, >?} >? >?void >-RebuildCkConstraints::alter(struct alter_space *alter) >+RebuildCkConstraints::alter(struct alter_space *alter) noexcept >?{ >? space_swap_ck_constraint(alter->old_space, alter->new_space); >?} >? >?void >-RebuildCkConstraints::rollback(struct alter_space *alter) >+RebuildCkConstraints::rollback(struct alter_space *alter) noexcept >?{ >? space_swap_ck_constraint(alter->new_space, alter->old_space); >?} >? >-RebuildCkConstraints::~RebuildCkConstraints() >+RebuildCkConstraints::~RebuildCkConstraints() noexcept >?{ >? struct ck_constraint *old_ck_constraint, *tmp; >? rlist_foreach_entry_safe(old_ck_constraint, &ck_constraint, link, tmp) { >@@ -1754,8 +1751,8 @@ class MoveCkConstraints: public AlterSpaceOp >? struct space *new_space); >?public: >? MoveCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter) {} >- virtual void alter(struct alter_space *alter); >- virtual void rollback(struct alter_space *alter); >+ void alter(struct alter_space *alter) noexcept override; >+ void rollback(struct alter_space *alter) noexcept override; >?}; >? >?void >@@ -1769,13 +1766,13 @@ MoveCkConstraints::space_swap_ck_constraint(struct space *old_space, >?} >? >?void >-MoveCkConstraints::alter(struct alter_space *alter) >+MoveCkConstraints::alter(struct alter_space *alter) noexcept >?{ >? space_swap_ck_constraint(alter->old_space, alter->new_space); >?} >? >?void >-MoveCkConstraints::rollback(struct alter_space *alter) >+MoveCkConstraints::rollback(struct alter_space *alter) noexcept >?{ >? space_swap_ck_constraint(alter->new_space, alter->old_space); >?} >@@ -1834,11 +1831,12 @@ public: >? if (new_id == NULL) >? diag_raise(); >? } >- virtual void prepare(struct alter_space *alter); >- virtual void alter(struct alter_space *alter); >- virtual void rollback(struct alter_space *alter); >- virtual void commit(struct alter_space *alter, int64_t signature); >- virtual ~CreateConstraintID(); >+ void prepare(struct alter_space *alter) override; >+ void alter(struct alter_space *alter) noexcept override; >+ void rollback(struct alter_space *alter) noexcept override; >+ void commit(struct alter_space *alter, >+ int64_t signature) noexcept override; >+ ~CreateConstraintID() noexcept override; >?}; >? >?void >@@ -1850,7 +1848,7 @@ CreateConstraintID::prepare(struct alter_space *alter) >?} >? >?void >-CreateConstraintID::alter(struct alter_space *alter) >+CreateConstraintID::alter(struct alter_space *alter) noexcept >?{ >? /* Alter() can't fail, so can't just throw an error. */ >? if (space_add_constraint_id(alter->old_space, new_id) != 0) >@@ -1858,14 +1856,15 @@ CreateConstraintID::alter(struct alter_space *alter) >?} >? >?void >-CreateConstraintID::rollback(struct alter_space *alter) >+CreateConstraintID::rollback(struct alter_space *alter) noexcept >?{ >? space_delete_constraint_id(alter->new_space, new_id->name); >? new_id = NULL; >?} >? >?void >-CreateConstraintID::commit(struct alter_space *alter, int64_t signature) >+CreateConstraintID::commit(struct alter_space *alter, >+ int64_t signature) noexcept >?{ >? (void) alter; >? (void) signature; >@@ -1876,7 +1875,7 @@ CreateConstraintID::commit(struct alter_space *alter, int64_t signature) >? new_id = NULL; >?} >? >-CreateConstraintID::~CreateConstraintID() >+CreateConstraintID::~CreateConstraintID() noexcept >?{ >? if (new_id != NULL) >? constraint_id_delete(new_id); >@@ -1891,19 +1890,20 @@ public: >? DropConstraintID(struct alter_space *alter, const char *name) >? :AlterSpaceOp(alter), old_id(NULL), name(name) >? {} >- virtual void alter(struct alter_space *alter); >- virtual void commit(struct alter_space *alter , int64_t signature); >- virtual void rollback(struct alter_space *alter); >+ void alter(struct alter_space *alter) noexcept override; >+ void commit(struct alter_space *alter, >+ int64_t signature) noexcept override; >+ void rollback(struct alter_space *alter) noexcept override; >?}; >? >?void >-DropConstraintID::alter(struct alter_space *alter) >+DropConstraintID::alter(struct alter_space *alter) noexcept >?{ >? old_id = space_pop_constraint_id(alter->old_space, name); >?} >? >?void >-DropConstraintID::commit(struct alter_space *alter, int64_t signature) >+DropConstraintID::commit(struct alter_space *alter, int64_t signature) noexcept >?{ >? (void) alter; >? (void) signature; >@@ -1911,7 +1911,7 @@ DropConstraintID::commit(struct alter_space *alter, int64_t signature) >?} >? >?void >-DropConstraintID::rollback(struct alter_space *alter) >+DropConstraintID::rollback(struct alter_space *alter) noexcept >?{ >? if (space_add_constraint_id(alter->new_space, old_id) != 0) { >? panic("Can't recover after constraint drop rollback (out of " >-- >2.7.4 >? ? ? -- Ilya Kosarev ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From i.kosarev at tarantool.org Tue Aug 4 13:04:04 2020 From: i.kosarev at tarantool.org (=?UTF-8?B?SWx5YSBLb3NhcmV2?=) Date: Tue, 04 Aug 2020 13:04:04 +0300 Subject: [Tarantool-patches] =?utf-8?q?=5BPATCH_v2_1/3=5D_alter=3A_use_goo?= =?utf-8?q?d_c++_style?= In-Reply-To: <1596534955.779570951@f731.i.mail.ru> References: <1594649887-15890-1-git-send-email-alyapunov@tarantool.org> <1594649887-15890-2-git-send-email-alyapunov@tarantool.org> <1596534955.779570951@f731.i.mail.ru> Message-ID: <1596535444.410127167@f712.i.mail.ru> Also there are 4 places where tabs were replaced by spaces, marked them below. >???????, 4 ??????? 2020, 12:55 +03:00 ?? Ilya Kosarev : >? >Hi! > >Yes, this?patch seems to be nice. >? >See 1 comment below. >>???????????, 13 ???? 2020, 17:18 +03:00 ?? Aleksandr Lyapunov < alyapunov at tarantool.org >: >>? >>We should use noexcept where it's appropriate. >>1.It informs a developer he could not care about exception safety >>2.It instructs a compiler to check throws if possible >> >>We should override for compile-time check of misprints in names >>of methods and types of arguments. >> >>Part of #5153 >>--- >>?src/box/alter.cc | 180 +++++++++++++++++++++++++++---------------------------- >>?1 file changed, 90 insertions(+), 90 deletions(-) >> >>diff --git a/src/box/alter.cc b/src/box/alter.cc >>index 249cee2..15a51d0 100644 >>--- a/src/box/alter.cc >>+++ b/src/box/alter.cc >>@@ -765,7 +765,7 @@ struct alter_space; >>? >>?class AlterSpaceOp { >>?public: >>- AlterSpaceOp(struct alter_space *alter); >>+ explicit AlterSpaceOp(struct alter_space *alter); >>? >>? /** Link in alter_space::ops. */ >>? struct rlist link; >>@@ -774,7 +774,7 @@ public: >>? * the space definition and/or key list that will be used >>? * for creating the new space. Must not yield or fail. >>? */ >>- virtual void alter_def(struct alter_space * /* alter */) {} >>+ virtual void alter_def(struct alter_space * /* alter */) noexcept {} >>? /** >>? * Called after creating a new space. Used for performing >>? * long-lasting operations, such as index rebuild or format >>@@ -788,21 +788,21 @@ public: >>? * state to the new space (e.g. move unchanged indexes). >>? * Must not yield or fail. >>? */ >>- virtual void alter(struct alter_space * /* alter */) {} >>+ virtual void alter(struct alter_space * /* alter */) noexcept {} >>? /** >>? * Called after the change has been successfully written >>? * to WAL. Must not fail. >>? */ >>? virtual void commit(struct alter_space * /* alter */, >>- int64_t /* signature */) {} >>+ int64_t /* signature */) noexcept {} >>? /** >>? * Called in case a WAL error occurred. It is supposed to undo >>? * the effect of AlterSpaceOp::prepare and AlterSpaceOp::alter. >>? * Must not fail. >>? */ >>- virtual void rollback(struct alter_space * /* alter */) {} >>+ virtual void rollback(struct alter_space * /* alter */) noexcept {} >>? >>- virtual ~AlterSpaceOp() {} >>+ virtual ~AlterSpaceOp() noexcept {} >>? >>? void *operator new(size_t size) >>? { >>@@ -984,7 +984,7 @@ struct mh_i32_t *AlterSpaceLock::registry; >>??* Replace the old space with a new one in the space cache. >>??*/ >>?static int >>-alter_space_commit(struct trigger *trigger, void *event) >>+alter_space_commit(struct trigger *trigger, void *event) noexcept >>?{ >>? struct txn *txn = (struct txn *) event; >>? struct alter_space *alter = (struct alter_space *) trigger->data; >>@@ -1002,12 +1002,8 @@ alter_space_commit(struct trigger *trigger, void *event) >>? * indexes into their new places. >>? */ >>? class AlterSpaceOp *op; >>- try { >>- rlist_foreach_entry(op, &alter->ops, link) >>- op->commit(alter, signature); >>- } catch (Exception *e) { >>- return -1; >>- } >>+ rlist_foreach_entry(op, &alter->ops, link) >>+ op->commit(alter, signature); >I guess this change should be in >?alter: do not catch exceptions when it's not necessary? commit. >>? >>? alter->new_space = NULL; /* for alter_space_delete(). */ >>? /* >>@@ -1029,7 +1025,7 @@ alter_space_commit(struct trigger *trigger, void *event) >>??* alter_space_commit() failure (unlikely) >>??*/ >>?static int >>-alter_space_rollback(struct trigger *trigger, void * /* event */) >>+alter_space_rollback(struct trigger *trigger, void * /* event */) noexcept >>?{ >>? struct alter_space *alter = (struct alter_space *) trigger->data; >>? /* Rollback alter ops */ >>@@ -1188,9 +1184,9 @@ alter_space_do(struct txn_stmt *stmt, struct alter_space *alter) >>?class CheckSpaceFormat: public AlterSpaceOp >>?{ >>?public: >>- CheckSpaceFormat(struct alter_space *alter) >>+ explicit CheckSpaceFormat(struct alter_space *alter) >>? :AlterSpaceOp(alter) {} >>- virtual void prepare(struct alter_space *alter); >>+ void prepare(struct alter_space *alter) override; >>?}; >>? >>?void >>@@ -1222,15 +1218,15 @@ public: >>? * names into an old dictionary and deletes new one. >>? */ >>? struct tuple_dictionary *new_dict; >>- virtual void alter_def(struct alter_space *alter); >>- virtual void alter(struct alter_space *alter); >>- virtual void rollback(struct alter_space *alter); >>- virtual ~ModifySpace(); >>+ void alter_def(struct alter_space *alter) noexcept override; >>+ void alter(struct alter_space *alter) noexcept override; >>+ void rollback(struct alter_space *alter) noexcept override; >>+ ~ModifySpace() noexcept override; >>?}; >>? >>?/** Amend the definition of the new space. */ >>?void >>-ModifySpace::alter_def(struct alter_space *alter) >>+ModifySpace::alter_def(struct alter_space *alter) noexcept >>?{ >>? /* >>? * Use the old dictionary for the new space, because >>@@ -1249,7 +1245,7 @@ ModifySpace::alter_def(struct alter_space *alter) >>?} >>? >>?void >>-ModifySpace::alter(struct alter_space *alter) >>+ModifySpace::alter(struct alter_space *alter) noexcept >>?{ >>? /* >>? * Move new names into an old dictionary, which already is >>@@ -1260,12 +1256,12 @@ ModifySpace::alter(struct alter_space *alter) >>?} >>? >>?void >>-ModifySpace::rollback(struct alter_space *alter) >>+ModifySpace::rollback(struct alter_space *alter) noexcept >>?{ >>? tuple_dictionary_swap(alter->new_space->def->dict, new_dict); >>?} >>? >>-ModifySpace::~ModifySpace() >>+ModifySpace::~ModifySpace() noexcept >>?{ >>? if (new_dict != NULL) >>? tuple_dictionary_unref(new_dict); >>@@ -1281,9 +1277,9 @@ public: >>? DropIndex(struct alter_space *alter, struct index *index) >>? :AlterSpaceOp(alter), old_index(index) {} >>? struct index *old_index; >>- virtual void alter_def(struct alter_space *alter); >>- virtual void prepare(struct alter_space *alter); >>- virtual void commit(struct alter_space *alter, int64_t lsn); >>+ void alter_def(struct alter_space *alter) noexcept override; >>+ void prepare(struct alter_space *alter) override; >>+ void commit(struct alter_space *alter, int64_t lsn) noexcept override; >>?}; >>? >>?/* >>@@ -1291,7 +1287,7 @@ public: >>??* the new index from it. >>??*/ >>?void >>-DropIndex::alter_def(struct alter_space * /* alter */) >>+DropIndex::alter_def(struct alter_space * /* alter */) noexcept >>?{ >>? rlist_del_entry(old_index->def, link); >>?} >>@@ -1305,7 +1301,7 @@ DropIndex::prepare(struct alter_space *alter) >>?} >>? >>?void >>-DropIndex::commit(struct alter_space *alter, int64_t signature) >>+DropIndex::commit(struct alter_space *alter, int64_t signature) noexcept >>?{ >>? (void)alter; >>? index_commit_drop(old_index, signature); >>@@ -1323,18 +1319,18 @@ public: >>? :AlterSpaceOp(alter), iid(iid_arg) {} >>? /** id of the index on the move. */ >>? uint32_t iid; >>- virtual void alter(struct alter_space *alter); >>- virtual void rollback(struct alter_space *alter); >>+ void alter(struct alter_space *alter) noexcept override; >>+ void rollback(struct alter_space *alter) noexcept override; >>?}; >>? >>?void >>-MoveIndex::alter(struct alter_space *alter) >>+MoveIndex::alter(struct alter_space *alter) noexcept >>?{ >>? space_swap_index(alter->old_space, alter->new_space, iid, iid); >>?} >>? >>?void >>-MoveIndex::rollback(struct alter_space *alter) >>+MoveIndex::rollback(struct alter_space *alter) noexcept >>?{ >>? space_swap_index(alter->old_space, alter->new_space, iid, iid); >>?} >>@@ -1365,23 +1361,23 @@ public: >>? struct index *old_index; >>? struct index *new_index; >>? struct index_def *new_index_def; >>- virtual void alter_def(struct alter_space *alter); >>- virtual void alter(struct alter_space *alter); >>- virtual void commit(struct alter_space *alter, int64_t lsn); >>- virtual void rollback(struct alter_space *alter); >>- virtual ~ModifyIndex(); >>+ void alter_def(struct alter_space *alter) noexcept override; >>+ void alter(struct alter_space *alter) noexcept override; >>+ void commit(struct alter_space *alter, int64_t lsn) noexcept override; >>+ void rollback(struct alter_space *alter) noexcept override; >>+ ~ModifyIndex() noexcept override; >>?}; >>? >>?/** Update the definition of the new space */ >>?void >>-ModifyIndex::alter_def(struct alter_space *alter) >>+ModifyIndex::alter_def(struct alter_space *alter) noexcept >>?{ >>? rlist_del_entry(old_index->def, link); >>? index_def_list_add(&alter->key_list, new_index_def); >>?} >>? >>?void >>-ModifyIndex::alter(struct alter_space *alter) >>+ModifyIndex::alter(struct alter_space *alter) noexcept >>?{ >>? new_index = space_index(alter->new_space, new_index_def->iid); >>? assert(old_index->def->iid == new_index->def->iid); >>@@ -1397,14 +1393,14 @@ ModifyIndex::alter(struct alter_space *alter) >>?} >>? >>?void >>-ModifyIndex::commit(struct alter_space *alter, int64_t signature) >>+ModifyIndex::commit(struct alter_space *alter, int64_t signature) noexcept >>?{ >>? (void)alter; >>? index_commit_modify(new_index, signature); >>?} >>? >>?void >>-ModifyIndex::rollback(struct alter_space *alter) >>+ModifyIndex::rollback(struct alter_space *alter) noexcept >>?{ >>? /* >>? * Restore indexes. >>@@ -1416,7 +1412,7 @@ ModifyIndex::rollback(struct alter_space *alter) >>? index_update_def(old_index); >>?} >>? >>-ModifyIndex::~ModifyIndex() >>+ModifyIndex::~ModifyIndex() noexcept >>?{ >>? index_def_delete(new_index_def); >>?} >>@@ -1432,15 +1428,15 @@ public: >>? CreateIndex(struct alter_space *alter, struct index_def *def) >>? :AlterSpaceOp(alter), new_index(NULL), new_index_def(def) >>? {} >>- virtual void alter_def(struct alter_space *alter); >>- virtual void prepare(struct alter_space *alter); >>- virtual void commit(struct alter_space *alter, int64_t lsn); >>- virtual ~CreateIndex(); >>+ void alter_def(struct alter_space *alter) noexcept override; >>+ void prepare(struct alter_space *alter) override; >>+ void commit(struct alter_space *alter, int64_t lsn) noexcept override; >>+ ~CreateIndex() noexcept override; >>?}; >>? >>?/** Add definition of the new key to the new space def. */ >>?void >>-CreateIndex::alter_def(struct alter_space *alter) >>+CreateIndex::alter_def(struct alter_space *alter) noexcept >>?{ >>? index_def_list_add(&alter->key_list, new_index_def); >>?} >>@@ -1480,7 +1476,7 @@ CreateIndex::prepare(struct alter_space *alter) >>?} >>? >>?void >>-CreateIndex::commit(struct alter_space *alter, int64_t signature) >>+CreateIndex::commit(struct alter_space *alter, int64_t signature) noexcept >>?{ >>? (void) alter; >>? assert(new_index != NULL); >>@@ -1488,7 +1484,7 @@ CreateIndex::commit(struct alter_space *alter, int64_t signature) >>? new_index = NULL; >>?} >>? >>-CreateIndex::~CreateIndex() >>+CreateIndex::~CreateIndex() noexcept >>?{ >>? if (new_index != NULL) >>? index_abort_create(new_index); >>@@ -1521,15 +1517,15 @@ public: >>? struct index_def *new_index_def; >>? /** Old index index_def. */ >>? struct index_def *old_index_def; >>- virtual void alter_def(struct alter_space *alter); >>- virtual void prepare(struct alter_space *alter); >>- virtual void commit(struct alter_space *alter, int64_t signature); >>- virtual ~RebuildIndex(); >>+ void alter_def(struct alter_space *alter) noexcept override; >>+ void prepare(struct alter_space *alter) override; >>+ void commit(struct alter_space *alter, int64_t signature) noexcept override; >>+ ~RebuildIndex() noexcept override; >>?}; >>? >>?/** Add definition of the new key to the new space def. */ >>?void >>-RebuildIndex::alter_def(struct alter_space *alter) >>+RebuildIndex::alter_def(struct alter_space *alter) noexcept >>?{ >>? rlist_del_entry(old_index_def, link); >>? index_def_list_add(&alter->key_list, new_index_def); >>@@ -1545,7 +1541,7 @@ RebuildIndex::prepare(struct alter_space *alter) >>?} >>? >>?void >>-RebuildIndex::commit(struct alter_space *alter, int64_t signature) >>+RebuildIndex::commit(struct alter_space *alter, int64_t signature) noexcept >>?{ >>? struct index *old_index = space_index(alter->old_space, >>? old_index_def->iid); >>@@ -1556,7 +1552,7 @@ RebuildIndex::commit(struct alter_space *alter, int64_t signature) >>? new_index = NULL; >>?} >>? >>-RebuildIndex::~RebuildIndex() >>+RebuildIndex::~RebuildIndex() noexcept >>?{ >>? if (new_index != NULL) >>? index_abort_create(new_index); >>@@ -1600,9 +1596,10 @@ public: >>? TruncateIndex(struct alter_space *alter, uint32_t iid) >>? : AlterSpaceOp(alter), iid(iid), >>? old_index(NULL), new_index(NULL) {} >>- virtual void prepare(struct alter_space *alter); >>- virtual void commit(struct alter_space *alter, int64_t signature); >>- virtual ~TruncateIndex(); >>+ void prepare(struct alter_space *alter) override; >>+ void commit(struct alter_space *alter, >>+ int64_t signature) noexcept override; * Here on a new line. >>+ ~TruncateIndex() noexcept override; >>?}; >>? >>?void >>@@ -1632,7 +1629,7 @@ TruncateIndex::prepare(struct alter_space *alter) >>?} >>? >>?void >>-TruncateIndex::commit(struct alter_space *alter, int64_t signature) >>+TruncateIndex::commit(struct alter_space *alter, int64_t signature) noexcept >>?{ >>? (void)alter; >>? index_commit_drop(old_index, signature); >>@@ -1640,7 +1637,7 @@ TruncateIndex::commit(struct alter_space *alter, int64_t signature) >>? new_index = NULL; >>?} >>? >>-TruncateIndex::~TruncateIndex() >>+TruncateIndex::~TruncateIndex() noexcept >>?{ >>? if (new_index == NULL) >>? return; >>@@ -1657,11 +1654,11 @@ class UpdateSchemaVersion: public AlterSpaceOp >>?public: >>? UpdateSchemaVersion(struct alter_space * alter) >>? :AlterSpaceOp(alter) {} >>- virtual void alter(struct alter_space *alter); >>+ void alter(struct alter_space *alter) noexcept override; >>?}; >>? >>?void >>-UpdateSchemaVersion::alter(struct alter_space *alter) >>+UpdateSchemaVersion::alter(struct alter_space *alter) noexcept >>?{ >>?????(void)alter; >>?????++schema_version; >>@@ -1684,10 +1681,10 @@ public: >>? RebuildCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter), >>? ck_constraint(RLIST_HEAD_INITIALIZER(ck_constraint)) {} >>? struct rlist ck_constraint; >>- virtual void prepare(struct alter_space *alter); >>- virtual void alter(struct alter_space *alter); >>- virtual void rollback(struct alter_space *alter); >>- virtual ~RebuildCkConstraints(); >>+ void prepare(struct alter_space *alter) override; >>+ void alter(struct alter_space *alter) noexcept override; >>+ void rollback(struct alter_space *alter) noexcept override; >>+ ~RebuildCkConstraints() noexcept override; >>?}; >>? >>?void >>@@ -1716,18 +1713,18 @@ RebuildCkConstraints::space_swap_ck_constraint(struct space *old_space, >>?} >>? >>?void >>-RebuildCkConstraints::alter(struct alter_space *alter) >>+RebuildCkConstraints::alter(struct alter_space *alter) noexcept >>?{ >>? space_swap_ck_constraint(alter->old_space, alter->new_space); >>?} >>? >>?void >>-RebuildCkConstraints::rollback(struct alter_space *alter) >>+RebuildCkConstraints::rollback(struct alter_space *alter) noexcept >>?{ >>? space_swap_ck_constraint(alter->new_space, alter->old_space); >>?} >>? >>-RebuildCkConstraints::~RebuildCkConstraints() >>+RebuildCkConstraints::~RebuildCkConstraints() noexcept >>?{ >>? struct ck_constraint *old_ck_constraint, *tmp; >>? rlist_foreach_entry_safe(old_ck_constraint, &ck_constraint, link, tmp) { >>@@ -1754,8 +1751,8 @@ class MoveCkConstraints: public AlterSpaceOp >>? struct space *new_space); >>?public: >>? MoveCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter) {} >>- virtual void alter(struct alter_space *alter); >>- virtual void rollback(struct alter_space *alter); >>+ void alter(struct alter_space *alter) noexcept override; >>+ void rollback(struct alter_space *alter) noexcept override; >>?}; >>? >>?void >>@@ -1769,13 +1766,13 @@ MoveCkConstraints::space_swap_ck_constraint(struct space *old_space, >>?} >>? >>?void >>-MoveCkConstraints::alter(struct alter_space *alter) >>+MoveCkConstraints::alter(struct alter_space *alter) noexcept >>?{ >>? space_swap_ck_constraint(alter->old_space, alter->new_space); >>?} >>? >>?void >>-MoveCkConstraints::rollback(struct alter_space *alter) >>+MoveCkConstraints::rollback(struct alter_space *alter) noexcept >>?{ >>? space_swap_ck_constraint(alter->new_space, alter->old_space); >>?} >>@@ -1834,11 +1831,12 @@ public: >>? if (new_id == NULL) >>? diag_raise(); >>? } >>- virtual void prepare(struct alter_space *alter); >>- virtual void alter(struct alter_space *alter); >>- virtual void rollback(struct alter_space *alter); >>- virtual void commit(struct alter_space *alter, int64_t signature); >>- virtual ~CreateConstraintID(); >>+ void prepare(struct alter_space *alter) override; >>+ void alter(struct alter_space *alter) noexcept override; >>+ void rollback(struct alter_space *alter) noexcept override; >>+ void commit(struct alter_space *alter, >>+ int64_t signature) noexcept override; * Here on a new line. >>+ ~CreateConstraintID() noexcept override; >>?}; >>? >>?void >>@@ -1850,7 +1848,7 @@ CreateConstraintID::prepare(struct alter_space *alter) >>?} >>? >>?void >>-CreateConstraintID::alter(struct alter_space *alter) >>+CreateConstraintID::alter(struct alter_space *alter) noexcept >>?{ >>? /* Alter() can't fail, so can't just throw an error. */ >>? if (space_add_constraint_id(alter->old_space, new_id) != 0) >>@@ -1858,14 +1856,15 @@ CreateConstraintID::alter(struct alter_space *alter) >>?} >>? >>?void >>-CreateConstraintID::rollback(struct alter_space *alter) >>+CreateConstraintID::rollback(struct alter_space *alter) noexcept >>?{ >>? space_delete_constraint_id(alter->new_space, new_id->name); >>? new_id = NULL; >>?} >>? >>?void >>-CreateConstraintID::commit(struct alter_space *alter, int64_t signature) >>+CreateConstraintID::commit(struct alter_space *alter, >>+ int64_t signature) noexcept * Here on a new line. >>?{ >>? (void) alter; >>? (void) signature; >>@@ -1876,7 +1875,7 @@ CreateConstraintID::commit(struct alter_space *alter, int64_t signature) >>? new_id = NULL; >>?} >>? >>-CreateConstraintID::~CreateConstraintID() >>+CreateConstraintID::~CreateConstraintID() noexcept >>?{ >>? if (new_id != NULL) >>? constraint_id_delete(new_id); >>@@ -1891,19 +1890,20 @@ public: >>? DropConstraintID(struct alter_space *alter, const char *name) >>? :AlterSpaceOp(alter), old_id(NULL), name(name) >>? {} >>- virtual void alter(struct alter_space *alter); >>- virtual void commit(struct alter_space *alter , int64_t signature); >>- virtual void rollback(struct alter_space *alter); >>+ void alter(struct alter_space *alter) noexcept override; >>+ void commit(struct alter_space *alter, >>+ int64_t signature) noexcept override; * Here on a new line. >>+ void rollback(struct alter_space *alter) noexcept override; >>?}; >>? >>?void >>-DropConstraintID::alter(struct alter_space *alter) >>+DropConstraintID::alter(struct alter_space *alter) noexcept >>?{ >>? old_id = space_pop_constraint_id(alter->old_space, name); >>?} >>? >>?void >>-DropConstraintID::commit(struct alter_space *alter, int64_t signature) >>+DropConstraintID::commit(struct alter_space *alter, int64_t signature) noexcept >>?{ >>? (void) alter; >>? (void) signature; >>@@ -1911,7 +1911,7 @@ DropConstraintID::commit(struct alter_space *alter, int64_t signature) >>?} >>? >>?void >>-DropConstraintID::rollback(struct alter_space *alter) >>+DropConstraintID::rollback(struct alter_space *alter) noexcept >>?{ >>? if (space_add_constraint_id(alter->new_space, old_id) != 0) { >>? panic("Can't recover after constraint drop rollback (out of " >>-- >>2.7.4 >>? >? >? >-- >Ilya Kosarev >? ? ? -- Ilya Kosarev ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From i.kosarev at tarantool.org Tue Aug 4 13:10:10 2020 From: i.kosarev at tarantool.org (=?UTF-8?B?SWx5YSBLb3NhcmV2?=) Date: Tue, 04 Aug 2020 13:10:10 +0300 Subject: [Tarantool-patches] =?utf-8?q?=5BPATCH_v2_2/3=5D_alter=3A_fix_cod?= =?utf-8?q?estyle?= In-Reply-To: <1594649887-15890-3-git-send-email-alyapunov@tarantool.org> References: <1594649887-15890-1-git-send-email-alyapunov@tarantool.org> <1594649887-15890-3-git-send-email-alyapunov@tarantool.org> Message-ID: <1596535810.142039609@f749.i.mail.ru> As far as i see, there are more places in alter.cc with wrong codestyle (spaces instead of tabs) but i guess those are the worst. ? >???????????, 13 ???? 2020, 17:18 +03:00 ?? Aleksandr Lyapunov : >? >--- >?src/box/alter.cc | 4 ++-- >?1 file changed, 2 insertions(+), 2 deletions(-) > >diff --git a/src/box/alter.cc b/src/box/alter.cc >index 15a51d0..9b2b8e8 100644 >--- a/src/box/alter.cc >+++ b/src/box/alter.cc >@@ -1660,8 +1660,8 @@ public: >?void >?UpdateSchemaVersion::alter(struct alter_space *alter) noexcept >?{ >- (void)alter; >- ++schema_version; >+ (void) alter; >+ ++schema_version; >?} >? >?/** >-- >2.7.4 >? ? ? -- Ilya Kosarev ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From v.shpilevoy at tarantool.org Wed Aug 5 00:04:11 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Tue, 4 Aug 2020 23:04:11 +0200 Subject: [Tarantool-patches] [PATCH vshard 1/1] storage: allow replica to boot before master Message-ID: <0963ccd10f91b249116916d5c1c91f5eef11e438.1596575031.git.v.shpilevoy@tarantool.org> Before the patch a replica couldn't call vshard.storage.cfg() before the master is bootstrapped. That could lead to an exception in the middle of vshard.storage.cfg(). It sounds not so bad and even looks unreachable, because replica bootstrap anyway is blocked in box.cfg() until a master node is started, otherwise the replica instance is terminated. But the box.cfg bootstrap != vshard bootstrap. It could be that both replica and master nodes are already bootstrapped and working, but vshard.storage.cfg() wasn't called yet. In that case vshard.storage.cfg() on the replica wouldn't block in box.cfg(), and would happily fail a few lines later in vshard.storage.cfg() at attempt to touch not existing box.space._bucket. That was the case for cartridge. The framework configures instances in its own way before vshard.storage.cfg(). Then it calls vshard.storage.cfg() on them in arbitrary order, and sometimes replicas were configured earlier than the master. The fail is fixed by simply skipping the _bucket's trigger installation on the replica node. Because so far it is not needed here for anything. The trigger's sole purpose is to increment bucket generation used only for DML on _bucket on the master node. Now vshard.storage.cfg() on replica is able to finish before master does the same. However the replica still won't be able to handle vshard.storage.* requests until receives vshard schema from master. Closes #237 --- Branch: http://github.com/tarantool/vshard/tree/gerold103/gh-237-boot-replica-first Issue: https://github.com/tarantool/vshard/issues/237 test/lua_libs/storage_template.lua | 55 +++++++++++- test/reload_evolution/storage.result | 10 +++ test/reload_evolution/storage.test.lua | 8 ++ test/router/boot_replica_first.result | 112 ++++++++++++++++++++++++ test/router/boot_replica_first.test.lua | 42 +++++++++ vshard/storage/init.lua | 18 +++- vshard/storage/reload_evolution.lua | 10 ++- 7 files changed, 251 insertions(+), 4 deletions(-) create mode 100644 test/router/boot_replica_first.result create mode 100644 test/router/boot_replica_first.test.lua diff --git a/test/lua_libs/storage_template.lua b/test/lua_libs/storage_template.lua index 29a753d..84e4180 100644 --- a/test/lua_libs/storage_template.lua +++ b/test/lua_libs/storage_template.lua @@ -1,5 +1,7 @@ #!/usr/bin/env tarantool +local luri = require('uri') + NAME = require('fio').basename(arg[0], '.lua') fiber = require('fiber') test_run = require('test_run').new() @@ -10,12 +12,63 @@ log = require('log') if not cfg.shard_index then cfg.shard_index = 'bucket_id' end +instance_uuid = util.name_to_uuid[NAME] + +-- +-- Bootstrap the instance exactly like vshard does. But don't +-- initialize any vshard-specific code. +-- +local function boot_like_vshard() + assert(type(box.cfg) == 'function') + for rs_uuid, rs in pairs(cfg.sharding) do + for replica_uuid, replica in pairs(rs.replicas) do + if replica_uuid == instance_uuid then + local box_cfg = {replication = {}} + box_cfg.instance_uuid = replica_uuid + box_cfg.replicaset_uuid = rs_uuid + box_cfg.listen = replica.uri + box_cfg.read_only = not replica.master + box_cfg.replication_connect_quorum = 0 + box_cfg.replication_timeout = 0.1 + for _, replica in pairs(rs.replicas) do + table.insert(box_cfg.replication, replica.uri) + end + box.cfg(box_cfg) + if not replica.master then + return + end + local uri = luri.parse(replica.uri) + box.schema.user.create(uri.login, { + password = uri.password, if_not_exists = true, + }) + box.schema.user.grant(uri.login, 'super') + return + end + end + end + assert(false) +end + +local omit_cfg = false +local i = 1 +while arg[i] ~= nil do + local key = arg[i] + i = i + 1 + if key == 'boot_before_cfg' then + boot_like_vshard() + omit_cfg = true + end +end vshard = require('vshard') echo_count = 0 cfg.replication_connect_timeout = 3 cfg.replication_timeout = 0.1 -vshard.storage.cfg(cfg, util.name_to_uuid[NAME]) + +if not omit_cfg then + vshard.storage.cfg(cfg, instance_uuid) +end + function bootstrap_storage(engine) box.once("testapp:schema:1", function() if rawget(_G, 'CHANGE_SPACE_IDS') then diff --git a/test/reload_evolution/storage.result b/test/reload_evolution/storage.result index ebf4fdc..4652c4f 100644 --- a/test/reload_evolution/storage.result +++ b/test/reload_evolution/storage.result @@ -92,6 +92,16 @@ test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ... vshard.storage.internal.reload_version --- +- 2 +... +-- +-- gh-237: should be only one trigger. During gh-237 the trigger installation +-- became conditional and therefore required to remember the current trigger +-- somewhere. When reloaded from the old version, the trigger needed to be +-- fetched from _bucket:on_replace(). +-- +#box.space._bucket:on_replace() +--- - 1 ... -- Make sure storage operates well. diff --git a/test/reload_evolution/storage.test.lua b/test/reload_evolution/storage.test.lua index 56c1693..06f7117 100644 --- a/test/reload_evolution/storage.test.lua +++ b/test/reload_evolution/storage.test.lua @@ -37,6 +37,14 @@ package.loaded['vshard.storage'] = nil vshard.storage = require("vshard.storage") test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil vshard.storage.internal.reload_version +-- +-- gh-237: should be only one trigger. During gh-237 the trigger installation +-- became conditional and therefore required to remember the current trigger +-- somewhere. When reloaded from the old version, the trigger needed to be +-- fetched from _bucket:on_replace(). +-- +#box.space._bucket:on_replace() + -- Make sure storage operates well. vshard.storage.bucket_force_drop(2000) vshard.storage.bucket_force_create(2000) diff --git a/test/router/boot_replica_first.result b/test/router/boot_replica_first.result new file mode 100644 index 0000000..1705230 --- /dev/null +++ b/test/router/boot_replica_first.result @@ -0,0 +1,112 @@ +-- test-run result file version 2 +test_run = require('test_run').new() + | --- + | ... +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } + | --- + | ... +test_run:create_cluster(REPLICASET_1, 'router', {args = 'boot_before_cfg'}) + | --- + | ... +util = require('util') + | --- + | ... +util.wait_master(test_run, REPLICASET_1, 'box_1_a') + | --- + | ... +_ = test_run:cmd("create server router with script='router/router_2.lua'") + | --- + | ... +_ = test_run:cmd("start server router") + | --- + | ... + +-- +-- gh-237: replica should be able to boot before master. Before the issue was +-- fixed, replica always tried to install a trigger on _bucket space even when +-- it was not created on a master yet - that lead to an exception in +-- storage.cfg. Now it should not install the trigger at all, because anyway it +-- is not needed on replica for anything. +-- + +test_run:switch('box_1_b') + | --- + | - true + | ... +vshard.storage.cfg(cfg, instance_uuid) + | --- + | ... +-- _bucket is not created yet. Will fail. +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) + | --- + | - attempt to index field '_bucket' (a nil value) + | ... + +test_run:switch('default') + | --- + | - true + | ... +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') + | --- + | ... + +test_run:switch('box_1_a') + | --- + | - true + | ... +vshard.storage.cfg(cfg, instance_uuid) + | --- + | ... + +test_run:switch('box_1_b') + | --- + | - true + | ... +test_run:wait_lsn('box_1_b', 'box_1_a') + | --- + | ... +-- Fails, but gracefully. Because the bucket is not found here. +vshard.storage.call(1, 'read', 'echo', {100}) + | --- + | - null + | - bucket_id: 1 + | reason: Not found + | code: 1 + | type: ShardingError + | message: 'Cannot perform action with bucket 1, reason: Not found' + | name: WRONG_BUCKET + | ... +-- Should not have triggers. +#box.space._bucket:on_replace() + | --- + | - 0 + | ... + +test_run:switch('router') + | --- + | - true + | ... +vshard.router.bootstrap() + | --- + | - true + | ... +vshard.router.callro(1, 'echo', {100}) + | --- + | - 100 + | ... + +test_run:switch("default") + | --- + | - true + | ... +test_run:cmd('stop server router') + | --- + | - true + | ... +test_run:cmd('delete server router') + | --- + | - true + | ... +test_run:drop_cluster(REPLICASET_1) + | --- + | ... diff --git a/test/router/boot_replica_first.test.lua b/test/router/boot_replica_first.test.lua new file mode 100644 index 0000000..7b1b3fd --- /dev/null +++ b/test/router/boot_replica_first.test.lua @@ -0,0 +1,42 @@ +test_run = require('test_run').new() +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } +test_run:create_cluster(REPLICASET_1, 'router', {args = 'boot_before_cfg'}) +util = require('util') +util.wait_master(test_run, REPLICASET_1, 'box_1_a') +_ = test_run:cmd("create server router with script='router/router_2.lua'") +_ = test_run:cmd("start server router") + +-- +-- gh-237: replica should be able to boot before master. Before the issue was +-- fixed, replica always tried to install a trigger on _bucket space even when +-- it was not created on a master yet - that lead to an exception in +-- storage.cfg. Now it should not install the trigger at all, because anyway it +-- is not needed on replica for anything. +-- + +test_run:switch('box_1_b') +vshard.storage.cfg(cfg, instance_uuid) +-- _bucket is not created yet. Will fail. +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) + +test_run:switch('default') +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') + +test_run:switch('box_1_a') +vshard.storage.cfg(cfg, instance_uuid) + +test_run:switch('box_1_b') +test_run:wait_lsn('box_1_b', 'box_1_a') +-- Fails, but gracefully. Because the bucket is not found here. +vshard.storage.call(1, 'read', 'echo', {100}) +-- Should not have triggers. +#box.space._bucket:on_replace() + +test_run:switch('router') +vshard.router.bootstrap() +vshard.router.callro(1, 'echo', {100}) + +test_run:switch("default") +test_run:cmd('stop server router') +test_run:cmd('delete server router') +test_run:drop_cluster(REPLICASET_1) diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua index c6a78fe..ed577f9 100644 --- a/vshard/storage/init.lua +++ b/vshard/storage/init.lua @@ -94,6 +94,17 @@ if not M then -- detect that _bucket was not changed between yields. -- bucket_generation = 0, + -- + -- Reference to the function used as on_replace trigger on + -- _bucket space. It is used to replace the trigger with + -- a new function when reload happens. It is kept + -- explicitly because the old function is deleted on + -- reload from the global namespace. On the other hand, it + -- is still stored in _bucket:on_replace() somewhere, but + -- it is not known where. The only 100% way to be able to + -- replace the old function is to keep its reference. + -- + bucket_on_replace = nil, ------------------- Garbage collection ------------------- -- Fiber to remove garbage buckets data. @@ -2435,8 +2446,11 @@ local function storage_cfg(cfg, this_replica_uuid, is_reload) local uri = luri.parse(this_replica.uri) schema_upgrade(is_master, uri.login, uri.password) - local old_trigger = box.space._bucket:on_replace()[1] - box.space._bucket:on_replace(bucket_generation_increment, old_trigger) + if is_master then + local old_trigger = M.bucket_on_replace + box.space._bucket:on_replace(bucket_generation_increment, old_trigger) + M.bucket_on_replace = bucket_generation_increment + end lreplicaset.rebind_replicasets(new_replicasets, M.replicasets) lreplicaset.outdate_replicasets(M.replicasets) diff --git a/vshard/storage/reload_evolution.lua b/vshard/storage/reload_evolution.lua index 5d09b11..f38af74 100644 --- a/vshard/storage/reload_evolution.lua +++ b/vshard/storage/reload_evolution.lua @@ -17,6 +17,14 @@ migrations[#migrations + 1] = function(M) -- Code to update Lua objects. end +migrations[#migrations + 1] = function(M) + local bucket = box.space._bucket + if bucket then + assert(M.bucket_on_replace == nil) + M.bucket_on_replace = bucket:on_replace()[1] + end +end + -- -- Perform an update based on a version stored in `M` (internals). -- @param M Old module internals which should be updated. @@ -33,7 +41,7 @@ local function upgrade(M) log.error(err_msg) error(err_msg) end - for i = start_version, #migrations do + for i = start_version + 1, #migrations do local ok, err = pcall(migrations[i], M) if ok then log.info('vshard.storage.reload_evolution: upgraded to %d version', -- 2.21.1 (Apple Git-122.3) From yaroslav.dynnikov at tarantool.org Wed Aug 5 01:45:25 2020 From: yaroslav.dynnikov at tarantool.org (Yaroslav Dynnikov) Date: Wed, 5 Aug 2020 01:45:25 +0300 Subject: [Tarantool-patches] [PATCH vshard 1/1] storage: allow replica to boot before master In-Reply-To: <0963ccd10f91b249116916d5c1c91f5eef11e438.1596575031.git.v.shpilevoy@tarantool.org> References: <0963ccd10f91b249116916d5c1c91f5eef11e438.1596575031.git.v.shpilevoy@tarantool.org> Message-ID: Hi, Vlad Thanks for the fix a lot. In general it looks good to me, but I've got few questions. Find them below. Best regards Yaroslav Dynnikov On Wed, 5 Aug 2020 at 00:04, Vladislav Shpilevoy wrote: > Before the patch a replica couldn't call vshard.storage.cfg() > before the master is bootstrapped. That could lead to an exception > in the middle of vshard.storage.cfg(). > > It sounds not so bad and even looks unreachable, because replica > bootstrap anyway is blocked in box.cfg() until a master node is > started, otherwise the replica instance is terminated. > But the box.cfg bootstrap != vshard bootstrap. > > It could be that both replica and master nodes are already > bootstrapped and working, but vshard.storage.cfg() wasn't called > yet. In that case vshard.storage.cfg() on the replica wouldn't > block in box.cfg(), and would happily fail a few lines later in > vshard.storage.cfg() at attempt to touch not existing > box.space._bucket. > > That was the case for cartridge. The framework configures > instances in its own way before vshard.storage.cfg(). Then it > calls vshard.storage.cfg() on them in arbitrary order, and > sometimes replicas were configured earlier than the master. > > The fail is fixed by simply skipping the _bucket's trigger > installation on the replica node. Because so far it is not needed > here for anything. The trigger's sole purpose is to increment > bucket generation used only for DML on _bucket on the master node. > > Now vshard.storage.cfg() on replica is able to finish before > master does the same. However the replica still won't be able to > handle vshard.storage.* requests until receives vshard schema > from master. > > Closes #237 > --- > Branch: > http://github.com/tarantool/vshard/tree/gerold103/gh-237-boot-replica-first > Issue: https://github.com/tarantool/vshard/issues/237 > > test/lua_libs/storage_template.lua | 55 +++++++++++- > test/reload_evolution/storage.result | 10 +++ > test/reload_evolution/storage.test.lua | 8 ++ > test/router/boot_replica_first.result | 112 ++++++++++++++++++++++++ > test/router/boot_replica_first.test.lua | 42 +++++++++ > vshard/storage/init.lua | 18 +++- > vshard/storage/reload_evolution.lua | 10 ++- > 7 files changed, 251 insertions(+), 4 deletions(-) > create mode 100644 test/router/boot_replica_first.result > create mode 100644 test/router/boot_replica_first.test.lua > > diff --git a/test/lua_libs/storage_template.lua > b/test/lua_libs/storage_template.lua > index 29a753d..84e4180 100644 > --- a/test/lua_libs/storage_template.lua > +++ b/test/lua_libs/storage_template.lua > @@ -1,5 +1,7 @@ > #!/usr/bin/env tarantool > > +local luri = require('uri') > + > NAME = require('fio').basename(arg[0], '.lua') > fiber = require('fiber') > test_run = require('test_run').new() > @@ -10,12 +12,63 @@ log = require('log') > if not cfg.shard_index then > cfg.shard_index = 'bucket_id' > end > +instance_uuid = util.name_to_uuid[NAME] > + > +-- > +-- Bootstrap the instance exactly like vshard does. But don't > +-- initialize any vshard-specific code. > +-- > +local function boot_like_vshard() > + assert(type(box.cfg) == 'function') > + for rs_uuid, rs in pairs(cfg.sharding) do > + for replica_uuid, replica in pairs(rs.replicas) do > + if replica_uuid == instance_uuid then > + local box_cfg = {replication = {}} > + box_cfg.instance_uuid = replica_uuid > + box_cfg.replicaset_uuid = rs_uuid > + box_cfg.listen = replica.uri > + box_cfg.read_only = not replica.master > + box_cfg.replication_connect_quorum = 0 > + box_cfg.replication_timeout = 0.1 > + for _, replica in pairs(rs.replicas) do > + table.insert(box_cfg.replication, replica.uri) > + end > + box.cfg(box_cfg) > + if not replica.master then > + return > + end > + local uri = luri.parse(replica.uri) > + box.schema.user.create(uri.login, { > + password = uri.password, if_not_exists = true, > + }) > + box.schema.user.grant(uri.login, 'super') > + return > + end > + end > + end > + assert(false) > Should there be some meaningful message? > +end > + > +local omit_cfg = false > +local i = 1 > +while arg[i] ~= nil do > + local key = arg[i] > + i = i + 1 > + if key == 'boot_before_cfg' then > + boot_like_vshard() > + omit_cfg = true > + end > +end > > vshard = require('vshard') > echo_count = 0 > cfg.replication_connect_timeout = 3 > cfg.replication_timeout = 0.1 > -vshard.storage.cfg(cfg, util.name_to_uuid[NAME]) > + > +if not omit_cfg then > + vshard.storage.cfg(cfg, instance_uuid) > +end > + > function bootstrap_storage(engine) > box.once("testapp:schema:1", function() > if rawget(_G, 'CHANGE_SPACE_IDS') then > diff --git a/test/reload_evolution/storage.result > b/test/reload_evolution/storage.result > index ebf4fdc..4652c4f 100644 > --- a/test/reload_evolution/storage.result > +++ b/test/reload_evolution/storage.result > @@ -92,6 +92,16 @@ test_run:grep_log('storage_2_a', > 'vshard.storage.reload_evolution: upgraded to') > ... > vshard.storage.internal.reload_version > --- > +- 2 > +... > +-- > +-- gh-237: should be only one trigger. During gh-237 the trigger > installation > +-- became conditional and therefore required to remember the current > trigger > +-- somewhere. When reloaded from the old version, the trigger needed to be > +-- fetched from _bucket:on_replace(). > +-- > +#box.space._bucket:on_replace() > +--- > - 1 > ... > -- Make sure storage operates well. > diff --git a/test/reload_evolution/storage.test.lua > b/test/reload_evolution/storage.test.lua > index 56c1693..06f7117 100644 > --- a/test/reload_evolution/storage.test.lua > +++ b/test/reload_evolution/storage.test.lua > @@ -37,6 +37,14 @@ package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') ~= nil > vshard.storage.internal.reload_version > +-- > +-- gh-237: should be only one trigger. During gh-237 the trigger > installation > +-- became conditional and therefore required to remember the current > trigger > +-- somewhere. When reloaded from the old version, the trigger needed to be > +-- fetched from _bucket:on_replace(). > +-- > +#box.space._bucket:on_replace() > + > -- Make sure storage operates well. > vshard.storage.bucket_force_drop(2000) > vshard.storage.bucket_force_create(2000) > diff --git a/test/router/boot_replica_first.result > b/test/router/boot_replica_first.result > new file mode 100644 > index 0000000..1705230 > --- /dev/null > +++ b/test/router/boot_replica_first.result > @@ -0,0 +1,112 @@ > +-- test-run result file version 2 > +test_run = require('test_run').new() > + | --- > + | ... > +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } > + | --- > + | ... > +test_run:create_cluster(REPLICASET_1, 'router', {args = > 'boot_before_cfg'}) > + | --- > + | ... > +util = require('util') > + | --- > + | ... > +util.wait_master(test_run, REPLICASET_1, 'box_1_a') > + | --- > + | ... > +_ = test_run:cmd("create server router with script='router/router_2.lua'") > + | --- > + | ... > +_ = test_run:cmd("start server router") > + | --- > + | ... > + > +-- > +-- gh-237: replica should be able to boot before master. Before the issue > was > +-- fixed, replica always tried to install a trigger on _bucket space even > when > +-- it was not created on a master yet - that lead to an exception in > +-- storage.cfg. Now it should not install the trigger at all, because > anyway it > +-- is not needed on replica for anything. > +-- > + > +test_run:switch('box_1_b') > + | --- > + | - true > + | ... > +vshard.storage.cfg(cfg, instance_uuid) > + | --- > + | ... > +-- _bucket is not created yet. Will fail. > +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) > + | --- > + | - attempt to index field '_bucket' (a nil value) > + | ... > + > +test_run:switch('default') > + | --- > + | - true > + | ... > +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') > + | --- > + | ... > + > +test_run:switch('box_1_a') > + | --- > + | - true > + | ... > +vshard.storage.cfg(cfg, instance_uuid) > + | --- > + | ... > + > +test_run:switch('box_1_b') > + | --- > + | - true > + | ... > +test_run:wait_lsn('box_1_b', 'box_1_a') > + | --- > + | ... > +-- Fails, but gracefully. Because the bucket is not found here. > +vshard.storage.call(1, 'read', 'echo', {100}) > + | --- > + | - null > + | - bucket_id: 1 > + | reason: Not found > + | code: 1 > + | type: ShardingError > + | message: 'Cannot perform action with bucket 1, reason: Not found' > + | name: WRONG_BUCKET > + | ... > +-- Should not have triggers. > +#box.space._bucket:on_replace() > + | --- > + | - 0 > + | ... > + > +test_run:switch('router') > + | --- > + | - true > + | ... > +vshard.router.bootstrap() > + | --- > + | - true > + | ... > +vshard.router.callro(1, 'echo', {100}) > + | --- > + | - 100 > + | ... > + > +test_run:switch("default") > + | --- > + | - true > + | ... > +test_run:cmd('stop server router') > + | --- > + | - true > + | ... > +test_run:cmd('delete server router') > + | --- > + | - true > + | ... > +test_run:drop_cluster(REPLICASET_1) > + | --- > + | ... > diff --git a/test/router/boot_replica_first.test.lua > b/test/router/boot_replica_first.test.lua > new file mode 100644 > index 0000000..7b1b3fd > --- /dev/null > +++ b/test/router/boot_replica_first.test.lua > @@ -0,0 +1,42 @@ > +test_run = require('test_run').new() > +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } > +test_run:create_cluster(REPLICASET_1, 'router', {args = > 'boot_before_cfg'}) > +util = require('util') > +util.wait_master(test_run, REPLICASET_1, 'box_1_a') > +_ = test_run:cmd("create server router with script='router/router_2.lua'") > +_ = test_run:cmd("start server router") > + > +-- > +-- gh-237: replica should be able to boot before master. Before the issue > was > +-- fixed, replica always tried to install a trigger on _bucket space even > when > +-- it was not created on a master yet - that lead to an exception in > +-- storage.cfg. Now it should not install the trigger at all, because > anyway it > +-- is not needed on replica for anything. > +-- > + > +test_run:switch('box_1_b') > +vshard.storage.cfg(cfg, instance_uuid) > +-- _bucket is not created yet. Will fail. > +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) > + > +test_run:switch('default') > +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') > + > +test_run:switch('box_1_a') > +vshard.storage.cfg(cfg, instance_uuid) > + > +test_run:switch('box_1_b') > +test_run:wait_lsn('box_1_b', 'box_1_a') > +-- Fails, but gracefully. Because the bucket is not found here. > +vshard.storage.call(1, 'read', 'echo', {100}) > +-- Should not have triggers. > +#box.space._bucket:on_replace() > + > +test_run:switch('router') > +vshard.router.bootstrap() > +vshard.router.callro(1, 'echo', {100}) > + > +test_run:switch("default") > +test_run:cmd('stop server router') > +test_run:cmd('delete server router') > +test_run:drop_cluster(REPLICASET_1) > diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua > index c6a78fe..ed577f9 100644 > --- a/vshard/storage/init.lua > +++ b/vshard/storage/init.lua > @@ -94,6 +94,17 @@ if not M then > -- detect that _bucket was not changed between yields. > -- > bucket_generation = 0, > + -- > + -- Reference to the function used as on_replace trigger on > + -- _bucket space. It is used to replace the trigger with > + -- a new function when reload happens. It is kept > + -- explicitly because the old function is deleted on > + -- reload from the global namespace. On the other hand, it > + -- is still stored in _bucket:on_replace() somewhere, but > + -- it is not known where. The only 100% way to be able to > + -- replace the old function is to keep its reference. > + -- > + bucket_on_replace = nil, > > ------------------- Garbage collection ------------------- > -- Fiber to remove garbage buckets data. > @@ -2435,8 +2446,11 @@ local function storage_cfg(cfg, this_replica_uuid, > is_reload) > local uri = luri.parse(this_replica.uri) > schema_upgrade(is_master, uri.login, uri.password) > > - local old_trigger = box.space._bucket:on_replace()[1] > - box.space._bucket:on_replace(bucket_generation_increment, old_trigger) > + if is_master then > + local old_trigger = M.bucket_on_replace > + box.space._bucket:on_replace(bucket_generation_increment, > old_trigger) > + M.bucket_on_replace = bucket_generation_increment > + end > > lreplicaset.rebind_replicasets(new_replicasets, M.replicasets) > lreplicaset.outdate_replicasets(M.replicasets) > diff --git a/vshard/storage/reload_evolution.lua > b/vshard/storage/reload_evolution.lua > index 5d09b11..f38af74 100644 > --- a/vshard/storage/reload_evolution.lua > +++ b/vshard/storage/reload_evolution.lua > @@ -17,6 +17,14 @@ migrations[#migrations + 1] = function(M) > -- Code to update Lua objects. > end > > +migrations[#migrations + 1] = function(M) > + local bucket = box.space._bucket > + if bucket then > + assert(M.bucket_on_replace == nil) > + M.bucket_on_replace = bucket:on_replace()[1] > + end > +end > + > -- > -- Perform an update based on a version stored in `M` (internals). > -- @param M Old module internals which should be updated. > @@ -33,7 +41,7 @@ local function upgrade(M) > log.error(err_msg) > error(err_msg) > end > - for i = start_version, #migrations do > + for i = start_version + 1, #migrations do > Is it already tested somewhere else (migrations I mean)? This change looks like a fix for some other problem. > local ok, err = pcall(migrations[i], M) > if ok then > log.info('vshard.storage.reload_evolution: upgraded to %d > version', > -- > 2.21.1 (Apple Git-122.3) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v.shpilevoy at tarantool.org Wed Aug 5 02:45:17 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Wed, 5 Aug 2020 01:45:17 +0200 Subject: [Tarantool-patches] [PATCH 0/2] JSON field multikey crash Message-ID: The patchset fixes 2 crashes related to multikey in JSON path tuple field access code. Also during working on this I found https://github.com/tarantool/tarantool/issues/5226, but couldn't find a simple solution. Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-5224-tuple-field-by-path-crash Issue: https://github.com/tarantool/tarantool/issues/5224 @ChangeLog * Fixed a crash when JSON tuple field access was used to get a multikey indexed field, and when a JSON contained [*] in the beginning; Vladislav Shpilevoy (2): tuple: fix multikey field JSON access crash tuple: fix access by JSON path starting from '[*]' src/box/tuple.c | 3 +- src/box/tuple.h | 8 + test/box/gh-5224-multikey-field-access.result | 164 ++++++++++++++++++ .../gh-5224-multikey-field-access.test.lua | 72 ++++++++ 4 files changed, 246 insertions(+), 1 deletion(-) create mode 100644 test/box/gh-5224-multikey-field-access.result create mode 100644 test/box/gh-5224-multikey-field-access.test.lua -- 2.21.1 (Apple Git-122.3) From v.shpilevoy at tarantool.org Wed Aug 5 02:45:18 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Wed, 5 Aug 2020 01:45:18 +0200 Subject: [Tarantool-patches] [PATCH 1/2] tuple: fix multikey field JSON access crash In-Reply-To: References: Message-ID: When a tuple had format with multikey indexes in it, any attempt to get a multikey indexed field by a JSON path from Lua led to a crash. That was because of incorrect interpretation of offset slot value in tuple's field map. Tuple field map is an array stored before the tuple's MessagePack data. Each element is a 4 byte offset to an indexed value to be able to get it for O(1) time without MessagePack decoding of all the previous fields. At least it was so before multikeys. Now tuple field map is not just an array. It is rather a 2-level array, somehow similar to ext4 FS. Some elements of the root array are positive numbers pointing at data. Some elements point at a second 'indirect' array, so called 'extra', size of which is individual for each tuple. These second arrays are used by multikey indexes to store offsets to each multikey indexed value in a tuple. It means, that if there is an offset slot, it can't be just used as is. It is allowed only if the field is not multikey. Otherwise it is neccessary to somehow get an index in the second 'indirect' array. This is what was happening - a multikey field was found, its offset slot was valid, but it was pointing at an 'indirect' array, not at the data. JSON tuple field access tried to use it as a data offset. The patch makes JSON field access degrade to fullscan when a field is multikey, but no multikey array index is provided. Closes #5224 --- src/box/tuple.h | 8 + test/box/gh-5224-multikey-field-access.result | 155 ++++++++++++++++++ .../gh-5224-multikey-field-access.test.lua | 66 ++++++++ 3 files changed, 229 insertions(+) create mode 100644 test/box/gh-5224-multikey-field-access.result create mode 100644 test/box/gh-5224-multikey-field-access.test.lua diff --git a/src/box/tuple.h b/src/box/tuple.h index 4752323e4..2b48d6af9 100644 --- a/src/box/tuple.h +++ b/src/box/tuple.h @@ -628,6 +628,14 @@ tuple_field_raw_by_path(struct tuple_format *format, const char *tuple, goto parse; if (offset_slot_hint != NULL) *offset_slot_hint = offset_slot; + /* + * When the field is multikey, the offset slot points not at the + * data. It points at 'extra' array of offsets for this multikey + * index. That array can only be accessed if index in that array + * is known. + */ + if (field->is_multikey_part && multikey_idx == MULTIKEY_NONE) + goto parse; offset_slot_access: /* Indexed field */ offset = field_map_get_offset(field_map, offset_slot, diff --git a/test/box/gh-5224-multikey-field-access.result b/test/box/gh-5224-multikey-field-access.result new file mode 100644 index 000000000..014e810d0 --- /dev/null +++ b/test/box/gh-5224-multikey-field-access.result @@ -0,0 +1,155 @@ +-- test-run result file version 2 +-- +-- gh-5224: tuple field access by JSON path crashed when tried to get a multikey +-- indexed field. +-- +format = {} + | --- + | ... +format[1] = {name = 'f1', type = 'unsigned'} + | --- + | ... +format[2] = {name = 'f2', type = 'array'} + | --- + | ... +s = box.schema.create_space('test', {format = format}) + | --- + | ... +_ = s:create_index('pk') + | --- + | ... +_ = s:create_index('sk', { \ + parts = { \ + {field = 2, path = "[*].tags", type = "unsigned"} \ + } \ +}) + | --- + | ... + +t = s:replace{1, {{tags = 2}}} + | --- + | ... +t['[2][1].tags'] + | --- + | - 2 + | ... + +t = s:replace{1, {{tags = 2}, {tags = 3}, {tags = 4}}} + | --- + | ... +t['[2]'] + | --- + | - [{'tags': 2}, {'tags': 3}, {'tags': 4}] + | ... +t['[2][1]'] + | --- + | - {'tags': 2} + | ... +t['[2][1].tags'] + | --- + | - 2 + | ... +t['[2][2]'] + | --- + | - {'tags': 3} + | ... +t['[2][2].tags'] + | --- + | - 3 + | ... +t['[2][3]'] + | --- + | - {'tags': 4} + | ... +t['[2][3].tags'] + | --- + | - 4 + | ... + +s:truncate() + | --- + | ... +s.index.sk:drop() + | --- + | ... +_ = s:create_index('sk', { \ + parts = { \ + {field = 2, path = "[*].p1.p2", type = "unsigned"} \ + } \ +}) + | --- + | ... + +t = s:replace{1, {{p1 = {p2 = 2}}}} + | --- + | ... +t['[2][1].p1.p2'] + | --- + | - 2 + | ... + +t = s:replace{1, { \ + { \ + p1 = { \ + p2 = 2, p3 = 3 \ + }, \ + p4 = 4 \ + }, \ + { \ + p1 = {p2 = 5} \ + }, \ + { \ + p1 = {p2 = 6} \ + } \ +}} + | --- + | ... + +t['[2][1].p1.p2'] + | --- + | - 2 + | ... +t['[2][1].p1.p3'] + | --- + | - 3 + | ... +t['[2][1].p1'] + | --- + | - {'p2': 2, 'p3': 3} + | ... +t['[2][1]'] + | --- + | - {'p4': 4, 'p1': {'p2': 2, 'p3': 3}} + | ... +t['[2][1].p4'] + | --- + | - 4 + | ... +t['[2][2].p1.p2'] + | --- + | - 5 + | ... +t['[2][2].p1'] + | --- + | - {'p2': 5} + | ... +t['[2][2]'] + | --- + | - {'p1': {'p2': 5}} + | ... +t['[2][3].p1.p2'] + | --- + | - 6 + | ... +t['[2][3].p1'] + | --- + | - {'p2': 6} + | ... +t['[2][3]'] + | --- + | - {'p1': {'p2': 6}} + | ... + +s:drop() + | --- + | ... diff --git a/test/box/gh-5224-multikey-field-access.test.lua b/test/box/gh-5224-multikey-field-access.test.lua new file mode 100644 index 000000000..19765171e --- /dev/null +++ b/test/box/gh-5224-multikey-field-access.test.lua @@ -0,0 +1,66 @@ +-- +-- gh-5224: tuple field access by JSON path crashed when tried to get a multikey +-- indexed field. +-- +format = {} +format[1] = {name = 'f1', type = 'unsigned'} +format[2] = {name = 'f2', type = 'array'} +s = box.schema.create_space('test', {format = format}) +_ = s:create_index('pk') +_ = s:create_index('sk', { \ + parts = { \ + {field = 2, path = "[*].tags", type = "unsigned"} \ + } \ +}) + +t = s:replace{1, {{tags = 2}}} +t['[2][1].tags'] + +t = s:replace{1, {{tags = 2}, {tags = 3}, {tags = 4}}} +t['[2]'] +t['[2][1]'] +t['[2][1].tags'] +t['[2][2]'] +t['[2][2].tags'] +t['[2][3]'] +t['[2][3].tags'] + +s:truncate() +s.index.sk:drop() +_ = s:create_index('sk', { \ + parts = { \ + {field = 2, path = "[*].p1.p2", type = "unsigned"} \ + } \ +}) + +t = s:replace{1, {{p1 = {p2 = 2}}}} +t['[2][1].p1.p2'] + +t = s:replace{1, { \ + { \ + p1 = { \ + p2 = 2, p3 = 3 \ + }, \ + p4 = 4 \ + }, \ + { \ + p1 = {p2 = 5} \ + }, \ + { \ + p1 = {p2 = 6} \ + } \ +}} + +t['[2][1].p1.p2'] +t['[2][1].p1.p3'] +t['[2][1].p1'] +t['[2][1]'] +t['[2][1].p4'] +t['[2][2].p1.p2'] +t['[2][2].p1'] +t['[2][2]'] +t['[2][3].p1.p2'] +t['[2][3].p1'] +t['[2][3]'] + +s:drop() -- 2.21.1 (Apple Git-122.3) From v.shpilevoy at tarantool.org Wed Aug 5 02:45:19 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Wed, 5 Aug 2020 01:45:19 +0200 Subject: [Tarantool-patches] [PATCH 2/2] tuple: fix access by JSON path starting from '[*]' In-Reply-To: References: Message-ID: <21871bd97383aa16c15d8bdad20775f2a85548ae.1596584571.git.v.shpilevoy@tarantool.org> Tuple JSON field access crashed when '[*]' was used as a first part of the JSON path. The patch makes it treated like 'field not found'. Follow-up #5224 --- src/box/tuple.c | 3 ++- test/box/gh-5224-multikey-field-access.result | 9 +++++++++ test/box/gh-5224-multikey-field-access.test.lua | 6 ++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/src/box/tuple.c b/src/box/tuple.c index 9f0f24c64..e77753a5b 100644 --- a/src/box/tuple.c +++ b/src/box/tuple.c @@ -531,7 +531,8 @@ tuple_field_raw_by_full_path(struct tuple_format *format, const char *tuple, break; } default: - assert(token.type == JSON_TOKEN_END); + assert(token.type == JSON_TOKEN_END || + token.type == JSON_TOKEN_ANY); return NULL; } return tuple_field_raw_by_path(format, tuple, field_map, fieldno, diff --git a/test/box/gh-5224-multikey-field-access.result b/test/box/gh-5224-multikey-field-access.result index 014e810d0..20df44cc3 100644 --- a/test/box/gh-5224-multikey-field-access.result +++ b/test/box/gh-5224-multikey-field-access.result @@ -150,6 +150,15 @@ t['[2][3]'] | - {'p1': {'p2': 6}} | ... +-- +-- Multikey path part could crash when used as a first part of the path during +-- accessing a tuple field. +-- +t['[*]'] + | --- + | - null + | ... + s:drop() | --- | ... diff --git a/test/box/gh-5224-multikey-field-access.test.lua b/test/box/gh-5224-multikey-field-access.test.lua index 19765171e..6f6691d3c 100644 --- a/test/box/gh-5224-multikey-field-access.test.lua +++ b/test/box/gh-5224-multikey-field-access.test.lua @@ -63,4 +63,10 @@ t['[2][3].p1.p2'] t['[2][3].p1'] t['[2][3]'] +-- +-- Multikey path part could crash when used as a first part of the path during +-- accessing a tuple field. +-- +t['[*]'] + s:drop() -- 2.21.1 (Apple Git-122.3) From yaroslav.dynnikov at tarantool.org Wed Aug 5 12:46:57 2020 From: yaroslav.dynnikov at tarantool.org (Yaroslav Dynnikov) Date: Wed, 5 Aug 2020 12:46:57 +0300 Subject: [Tarantool-patches] [PATCH vshard 1/1] storage: allow replica to boot before master In-Reply-To: References: <0963ccd10f91b249116916d5c1c91f5eef11e438.1596575031.git.v.shpilevoy@tarantool.org> Message-ID: And one more question. Best regards Yaroslav Dynnikov On Wed, 5 Aug 2020 at 01:45, Yaroslav Dynnikov < yaroslav.dynnikov at tarantool.org> wrote: > Hi, Vlad > > Thanks for the fix a lot. In general it looks good to me, but I've got few > questions. Find them below. > > Best regards > Yaroslav Dynnikov > > > On Wed, 5 Aug 2020 at 00:04, Vladislav Shpilevoy < > v.shpilevoy at tarantool.org> wrote: > >> Before the patch a replica couldn't call vshard.storage.cfg() >> before the master is bootstrapped. That could lead to an exception >> in the middle of vshard.storage.cfg(). >> >> It sounds not so bad and even looks unreachable, because replica >> bootstrap anyway is blocked in box.cfg() until a master node is >> started, otherwise the replica instance is terminated. >> But the box.cfg bootstrap != vshard bootstrap. >> >> It could be that both replica and master nodes are already >> bootstrapped and working, but vshard.storage.cfg() wasn't called >> yet. In that case vshard.storage.cfg() on the replica wouldn't >> block in box.cfg(), and would happily fail a few lines later in >> vshard.storage.cfg() at attempt to touch not existing >> box.space._bucket. >> >> That was the case for cartridge. The framework configures >> instances in its own way before vshard.storage.cfg(). Then it >> calls vshard.storage.cfg() on them in arbitrary order, and >> sometimes replicas were configured earlier than the master. >> >> The fail is fixed by simply skipping the _bucket's trigger >> installation on the replica node. Because so far it is not needed >> here for anything. The trigger's sole purpose is to increment >> bucket generation used only for DML on _bucket on the master node. >> >> Now vshard.storage.cfg() on replica is able to finish before >> master does the same. However the replica still won't be able to >> handle vshard.storage.* requests until receives vshard schema >> from master. >> >> Closes #237 >> --- >> Branch: >> http://github.com/tarantool/vshard/tree/gerold103/gh-237-boot-replica-first >> Issue: https://github.com/tarantool/vshard/issues/237 >> >> test/lua_libs/storage_template.lua | 55 +++++++++++- >> test/reload_evolution/storage.result | 10 +++ >> test/reload_evolution/storage.test.lua | 8 ++ >> test/router/boot_replica_first.result | 112 ++++++++++++++++++++++++ >> test/router/boot_replica_first.test.lua | 42 +++++++++ >> vshard/storage/init.lua | 18 +++- >> vshard/storage/reload_evolution.lua | 10 ++- >> 7 files changed, 251 insertions(+), 4 deletions(-) >> create mode 100644 test/router/boot_replica_first.result >> create mode 100644 test/router/boot_replica_first.test.lua >> >> diff --git a/test/lua_libs/storage_template.lua >> b/test/lua_libs/storage_template.lua >> index 29a753d..84e4180 100644 >> --- a/test/lua_libs/storage_template.lua >> +++ b/test/lua_libs/storage_template.lua >> @@ -1,5 +1,7 @@ >> #!/usr/bin/env tarantool >> >> +local luri = require('uri') >> + >> NAME = require('fio').basename(arg[0], '.lua') >> fiber = require('fiber') >> test_run = require('test_run').new() >> @@ -10,12 +12,63 @@ log = require('log') >> if not cfg.shard_index then >> cfg.shard_index = 'bucket_id' >> end >> +instance_uuid = util.name_to_uuid[NAME] >> + >> +-- >> +-- Bootstrap the instance exactly like vshard does. But don't >> +-- initialize any vshard-specific code. >> +-- >> +local function boot_like_vshard() >> + assert(type(box.cfg) == 'function') >> + for rs_uuid, rs in pairs(cfg.sharding) do >> + for replica_uuid, replica in pairs(rs.replicas) do >> + if replica_uuid == instance_uuid then >> + local box_cfg = {replication = {}} >> + box_cfg.instance_uuid = replica_uuid >> + box_cfg.replicaset_uuid = rs_uuid >> + box_cfg.listen = replica.uri >> + box_cfg.read_only = not replica.master >> + box_cfg.replication_connect_quorum = 0 >> + box_cfg.replication_timeout = 0.1 >> + for _, replica in pairs(rs.replicas) do >> + table.insert(box_cfg.replication, replica.uri) >> + end >> + box.cfg(box_cfg) >> + if not replica.master then >> + return >> + end >> + local uri = luri.parse(replica.uri) >> + box.schema.user.create(uri.login, { >> + password = uri.password, if_not_exists = true, >> + }) >> + box.schema.user.grant(uri.login, 'super') >> + return >> + end >> + end >> + end >> + assert(false) >> > > Should there be some meaningful message? > > >> +end >> + >> +local omit_cfg = false >> +local i = 1 >> +while arg[i] ~= nil do >> + local key = arg[i] >> + i = i + 1 >> + if key == 'boot_before_cfg' then >> + boot_like_vshard() >> + omit_cfg = true >> + end >> +end >> >> vshard = require('vshard') >> echo_count = 0 >> cfg.replication_connect_timeout = 3 >> cfg.replication_timeout = 0.1 >> -vshard.storage.cfg(cfg, util.name_to_uuid[NAME]) >> + >> +if not omit_cfg then >> + vshard.storage.cfg(cfg, instance_uuid) >> +end >> + >> function bootstrap_storage(engine) >> box.once("testapp:schema:1", function() >> if rawget(_G, 'CHANGE_SPACE_IDS') then >> diff --git a/test/reload_evolution/storage.result >> b/test/reload_evolution/storage.result >> index ebf4fdc..4652c4f 100644 >> --- a/test/reload_evolution/storage.result >> +++ b/test/reload_evolution/storage.result >> @@ -92,6 +92,16 @@ test_run:grep_log('storage_2_a', >> 'vshard.storage.reload_evolution: upgraded to') >> ... >> vshard.storage.internal.reload_version >> --- >> +- 2 >> +... >> +-- >> +-- gh-237: should be only one trigger. During gh-237 the trigger >> installation >> +-- became conditional and therefore required to remember the current >> trigger >> +-- somewhere. When reloaded from the old version, the trigger needed to >> be >> +-- fetched from _bucket:on_replace(). >> +-- >> +#box.space._bucket:on_replace() >> +--- >> - 1 >> ... >> -- Make sure storage operates well. >> diff --git a/test/reload_evolution/storage.test.lua >> b/test/reload_evolution/storage.test.lua >> index 56c1693..06f7117 100644 >> --- a/test/reload_evolution/storage.test.lua >> +++ b/test/reload_evolution/storage.test.lua >> @@ -37,6 +37,14 @@ package.loaded['vshard.storage'] = nil >> vshard.storage = require("vshard.storage") >> test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: >> upgraded to') ~= nil >> vshard.storage.internal.reload_version >> +-- >> +-- gh-237: should be only one trigger. During gh-237 the trigger >> installation >> +-- became conditional and therefore required to remember the current >> trigger >> +-- somewhere. When reloaded from the old version, the trigger needed to >> be >> +-- fetched from _bucket:on_replace(). >> +-- >> +#box.space._bucket:on_replace() >> + >> -- Make sure storage operates well. >> vshard.storage.bucket_force_drop(2000) >> vshard.storage.bucket_force_create(2000) >> diff --git a/test/router/boot_replica_first.result >> b/test/router/boot_replica_first.result >> new file mode 100644 >> index 0000000..1705230 >> --- /dev/null >> +++ b/test/router/boot_replica_first.result >> @@ -0,0 +1,112 @@ >> +-- test-run result file version 2 >> +test_run = require('test_run').new() >> + | --- >> + | ... >> +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } >> + | --- >> + | ... >> +test_run:create_cluster(REPLICASET_1, 'router', {args = >> 'boot_before_cfg'}) >> + | --- >> + | ... >> +util = require('util') >> + | --- >> + | ... >> +util.wait_master(test_run, REPLICASET_1, 'box_1_a') >> + | --- >> + | ... >> +_ = test_run:cmd("create server router with >> script='router/router_2.lua'") >> + | --- >> + | ... >> +_ = test_run:cmd("start server router") >> + | --- >> + | ... >> + >> +-- >> +-- gh-237: replica should be able to boot before master. Before the >> issue was >> +-- fixed, replica always tried to install a trigger on _bucket space >> even when >> +-- it was not created on a master yet - that lead to an exception in >> +-- storage.cfg. Now it should not install the trigger at all, because >> anyway it >> +-- is not needed on replica for anything. >> +-- >> + >> +test_run:switch('box_1_b') >> + | --- >> + | - true >> + | ... >> +vshard.storage.cfg(cfg, instance_uuid) >> + | --- >> + | ... >> +-- _bucket is not created yet. Will fail. >> +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) >> + | --- >> + | - attempt to index field '_bucket' (a nil value) >> + | ... >> + >> +test_run:switch('default') >> + | --- >> + | - true >> + | ... >> +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') >> + | --- >> + | ... >> + >> +test_run:switch('box_1_a') >> + | --- >> + | - true >> + | ... >> +vshard.storage.cfg(cfg, instance_uuid) >> + | --- >> + | ... >> + >> +test_run:switch('box_1_b') >> + | --- >> + | - true >> + | ... >> +test_run:wait_lsn('box_1_b', 'box_1_a') >> + | --- >> + | ... >> +-- Fails, but gracefully. Because the bucket is not found here. >> +vshard.storage.call(1, 'read', 'echo', {100}) >> + | --- >> + | - null >> + | - bucket_id: 1 >> + | reason: Not found >> + | code: 1 >> + | type: ShardingError >> + | message: 'Cannot perform action with bucket 1, reason: Not found' >> + | name: WRONG_BUCKET >> + | ... >> +-- Should not have triggers. >> +#box.space._bucket:on_replace() >> + | --- >> + | - 0 >> + | ... >> + >> +test_run:switch('router') >> + | --- >> + | - true >> + | ... >> +vshard.router.bootstrap() >> + | --- >> + | - true >> + | ... >> +vshard.router.callro(1, 'echo', {100}) >> + | --- >> + | - 100 >> + | ... >> + >> +test_run:switch("default") >> + | --- >> + | - true >> + | ... >> +test_run:cmd('stop server router') >> + | --- >> + | - true >> + | ... >> +test_run:cmd('delete server router') >> + | --- >> + | - true >> + | ... >> +test_run:drop_cluster(REPLICASET_1) >> + | --- >> + | ... >> diff --git a/test/router/boot_replica_first.test.lua >> b/test/router/boot_replica_first.test.lua >> new file mode 100644 >> index 0000000..7b1b3fd >> --- /dev/null >> +++ b/test/router/boot_replica_first.test.lua >> @@ -0,0 +1,42 @@ >> +test_run = require('test_run').new() >> +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } >> +test_run:create_cluster(REPLICASET_1, 'router', {args = >> 'boot_before_cfg'}) >> +util = require('util') >> +util.wait_master(test_run, REPLICASET_1, 'box_1_a') >> +_ = test_run:cmd("create server router with >> script='router/router_2.lua'") >> +_ = test_run:cmd("start server router") >> + >> +-- >> +-- gh-237: replica should be able to boot before master. Before the >> issue was >> +-- fixed, replica always tried to install a trigger on _bucket space >> even when >> +-- it was not created on a master yet - that lead to an exception in >> +-- storage.cfg. Now it should not install the trigger at all, because >> anyway it >> +-- is not needed on replica for anything. >> +-- >> + >> +test_run:switch('box_1_b') >> +vshard.storage.cfg(cfg, instance_uuid) >> +-- _bucket is not created yet. Will fail. >> +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) >> + >> +test_run:switch('default') >> +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') >> + >> +test_run:switch('box_1_a') >> +vshard.storage.cfg(cfg, instance_uuid) >> + >> +test_run:switch('box_1_b') >> +test_run:wait_lsn('box_1_b', 'box_1_a') >> +-- Fails, but gracefully. Because the bucket is not found here. >> +vshard.storage.call(1, 'read', 'echo', {100}) >> +-- Should not have triggers. >> +#box.space._bucket:on_replace() >> + >> +test_run:switch('router') >> +vshard.router.bootstrap() >> +vshard.router.callro(1, 'echo', {100}) >> + >> +test_run:switch("default") >> +test_run:cmd('stop server router') >> +test_run:cmd('delete server router') >> +test_run:drop_cluster(REPLICASET_1) >> diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua >> index c6a78fe..ed577f9 100644 >> --- a/vshard/storage/init.lua >> +++ b/vshard/storage/init.lua >> @@ -94,6 +94,17 @@ if not M then >> -- detect that _bucket was not changed between yields. >> -- >> bucket_generation = 0, >> + -- >> + -- Reference to the function used as on_replace trigger on >> + -- _bucket space. It is used to replace the trigger with >> + -- a new function when reload happens. It is kept >> + -- explicitly because the old function is deleted on >> + -- reload from the global namespace. On the other hand, it >> + -- is still stored in _bucket:on_replace() somewhere, but >> + -- it is not known where. The only 100% way to be able to >> + -- replace the old function is to keep its reference. >> + -- >> + bucket_on_replace = nil, >> >> ------------------- Garbage collection ------------------- >> -- Fiber to remove garbage buckets data. >> @@ -2435,8 +2446,11 @@ local function storage_cfg(cfg, this_replica_uuid, >> is_reload) >> local uri = luri.parse(this_replica.uri) >> schema_upgrade(is_master, uri.login, uri.password) >> >> - local old_trigger = box.space._bucket:on_replace()[1] >> - box.space._bucket:on_replace(bucket_generation_increment, >> old_trigger) >> + if is_master then >> + local old_trigger = M.bucket_on_replace >> + box.space._bucket:on_replace(bucket_generation_increment, >> old_trigger) >> + M.bucket_on_replace = bucket_generation_increment >> + end >> > It seems that the trigger is never removed. If a replica was a master some time ago, it still has the trigger. Is it safe? >> lreplicaset.rebind_replicasets(new_replicasets, M.replicasets) >> lreplicaset.outdate_replicasets(M.replicasets) >> diff --git a/vshard/storage/reload_evolution.lua >> b/vshard/storage/reload_evolution.lua >> index 5d09b11..f38af74 100644 >> --- a/vshard/storage/reload_evolution.lua >> +++ b/vshard/storage/reload_evolution.lua >> @@ -17,6 +17,14 @@ migrations[#migrations + 1] = function(M) >> -- Code to update Lua objects. >> end >> >> +migrations[#migrations + 1] = function(M) >> + local bucket = box.space._bucket >> + if bucket then >> + assert(M.bucket_on_replace == nil) >> + M.bucket_on_replace = bucket:on_replace()[1] >> + end >> +end >> + >> -- >> -- Perform an update based on a version stored in `M` (internals). >> -- @param M Old module internals which should be updated. >> @@ -33,7 +41,7 @@ local function upgrade(M) >> log.error(err_msg) >> error(err_msg) >> end >> - for i = start_version, #migrations do >> + for i = start_version + 1, #migrations do >> > > Is it already tested somewhere else (migrations I mean)? This change looks > like a fix for some other problem. > > >> local ok, err = pcall(migrations[i], M) >> if ok then >> log.info('vshard.storage.reload_evolution: upgraded to %d >> version', >> -- >> 2.21.1 (Apple Git-122.3) >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From i.kosarev at tarantool.org Wed Aug 5 13:12:51 2020 From: i.kosarev at tarantool.org (Ilya Kosarev) Date: Wed, 5 Aug 2020 13:12:51 +0300 Subject: [Tarantool-patches] [PATCH v2] tuple: drop extra restrictions for multikey index Message-ID: <20200805101251.30600-1-i.kosarev@tarantool.org> Multikey index did not work properly with nullable root field in tuple_raw_multikey_count(). Now it is fixed and corresponding restrictions are dropped. This also means that we can drop implicit nullability update for array/map fields and make all fields nullable by default, as it was until e1d3fe8ab8eed65394ad17409401a93b6fcdc435 (tuple format: don't allow null where array/map is expected), as far as default non-nullability itself doesn't solve any real problems while providing confusing behavior (gh-5027). Follow-up #5027 Closes #5192 --- Branch: https://github.com/tarantool/tarantool/tree/i.kosarev/gh-5192-fix-multikey-index-restrictions Issue: https://github.com/tarantool/tarantool/issues/5192 @ChangeLog: * Dropped restrictions on nullable multikey index root. All fields are now nullable by default as it was before 2.2.1 (gh-5192). Changes in v2: - removed insignificant changes - fixed tests comments src/box/memtx_space.c | 8 --- src/box/tuple.c | 5 +- src/box/tuple_format.c | 20 ------ src/box/vinyl.c | 8 --- test/engine/gh-5027-fields-nullability.result | 71 +++---------------- .../gh-5027-fields-nullability.test.lua | 35 +++------ .../gh-5192-multikey-root-nullability.result | 62 ++++++++++++++++ ...gh-5192-multikey-root-nullability.test.lua | 20 ++++++ test/engine/json.result | 20 +++--- test/engine/json.test.lua | 6 +- test/engine/multikey.result | 8 +-- test/engine/multikey.test.lua | 4 +- 12 files changed, 123 insertions(+), 144 deletions(-) create mode 100644 test/engine/gh-5192-multikey-root-nullability.result create mode 100644 test/engine/gh-5192-multikey-root-nullability.test.lua diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c index 8452ab430..d30ce44b8 100644 --- a/src/box/memtx_space.c +++ b/src/box/memtx_space.c @@ -604,14 +604,6 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def) return -1; } } - if (key_def->is_multikey && - key_def->multikey_fieldno < space->def->field_count && - space->def->fields[key_def->multikey_fieldno].is_nullable) { - diag_set(ClientError, ER_UNSUPPORTED, - "multikey index", - "nullable root field"); - return -1; - } switch (index_def->type) { case HASH: if (! index_def->opts.is_unique) { diff --git a/src/box/tuple.c b/src/box/tuple.c index 9f0f24c64..94a3a96ba 100644 --- a/src/box/tuple.c +++ b/src/box/tuple.c @@ -554,7 +554,10 @@ tuple_raw_multikey_count(struct tuple_format *format, const char *data, NULL, MULTIKEY_NONE); if (array_raw == NULL) return 0; - assert(mp_typeof(*array_raw) == MP_ARRAY); + enum mp_type type = mp_typeof(*array_raw); + if (type == MP_NIL) + return 0; + assert(type == MP_ARRAY); return mp_decode_array(&array_raw); } diff --git a/src/box/tuple_format.c b/src/box/tuple_format.c index bae6c67cd..9b817d3cf 100644 --- a/src/box/tuple_format.c +++ b/src/box/tuple_format.c @@ -242,24 +242,6 @@ tuple_field_ensure_child_compatibility(struct tuple_field *parent, return 0; } -/** - * Tuple fields are nullable by default. However, it is not ok - * for array/map fields, as far as their implicit nullability - * might break field accessors expectations, provide confusing - * error messages and cause incorrect behaviour of - * tuple_multikey_count(). Thus array/map fields have to be - * non-nullable by default, which means we have to update default - * nullability for them. - */ -static void -tuple_field_update_nullability(struct tuple_field *field) -{ - if ((field->type == FIELD_TYPE_ARRAY || - field->type == FIELD_TYPE_MAP) && - tuple_field_is_nullable(field)) - field->nullable_action = ON_CONFLICT_ACTION_DEFAULT; -} - /** * Given a field number and a path, add the corresponding field * to the tuple format, allocating intermediate fields if @@ -335,13 +317,11 @@ tuple_format_add_field(struct tuple_format *format, uint32_t fieldno, parent->offset_slot = *current_slot; } } - tuple_field_update_nullability(parent); parent->is_key_part = true; next->is_multikey_part = is_multikey; parent = next; token_count++; } - tuple_field_update_nullability(parent); /* * The path has already been verified by the * key_def_decode_parts function. diff --git a/src/box/vinyl.c b/src/box/vinyl.c index 32301d7ba..5fa3ea3a5 100644 --- a/src/box/vinyl.c +++ b/src/box/vinyl.c @@ -658,14 +658,6 @@ vinyl_space_check_index_def(struct space *space, struct index_def *index_def) diag_set(ClientError, ER_NULLABLE_PRIMARY, space_name(space)); return -1; } - if (key_def->is_multikey && - key_def->multikey_fieldno < space->def->field_count && - space->def->fields[key_def->multikey_fieldno].is_nullable) { - diag_set(ClientError, ER_UNSUPPORTED, - "multikey index", - "nullable root field"); - return -1; - } /* Check that there are no ANY, ARRAY, MAP parts */ for (uint32_t i = 0; i < key_def->part_count; i++) { struct key_part *part = &key_def->parts[i]; diff --git a/test/engine/gh-5027-fields-nullability.result b/test/engine/gh-5027-fields-nullability.result index 1121727f6..382944d53 100644 --- a/test/engine/gh-5027-fields-nullability.result +++ b/test/engine/gh-5027-fields-nullability.result @@ -12,19 +12,19 @@ _ = s:create_index('i1', {parts={{1, 'unsigned'}}}) _ = s:create_index('i2', {parts={{5, 'unsigned', is_nullable=true}}}) | --- | ... -s:replace{1} +s:replace{1} -- ok | --- | - [1] | ... -s:replace{1, box.NULL} +s:replace{1, box.NULL} -- ok | --- | - [1, null] | ... -s:replace{1, box.NULL, box.NULL} +s:replace{1, box.NULL, box.NULL} -- ok | --- | - [1, null, null] | ... -s:replace{1, box.NULL, box.NULL, box.NULL} +s:replace{1, box.NULL, box.NULL, box.NULL} -- ok | --- | - [1, null, null, null] | ... @@ -41,79 +41,26 @@ _ = s:create_index('i1', {parts={{1, 'unsigned'}}}) _ = s:create_index('i2', {parts={{5, 'unsigned', is_nullable=false}}}) | --- | ... -s:replace{1} +s:replace{1} -- error | --- | - error: Tuple field 5 required by space format is missing | ... -s:replace{1, box.NULL} +s:replace{1, box.NULL} -- error | --- | - error: Tuple field 5 required by space format is missing | ... -s:replace{1, box.NULL, box.NULL} +s:replace{1, box.NULL, box.NULL} -- error | --- | - error: Tuple field 5 required by space format is missing | ... -s:replace{1, box.NULL, box.NULL, box.NULL} +s:replace{1, box.NULL, box.NULL, box.NULL} -- error | --- | - error: Tuple field 5 required by space format is missing | ... -s:replace{1, box.NULL, box.NULL, box.NULL, 5} +s:replace{1, box.NULL, box.NULL, box.NULL, 5} -- ok | --- | - [1, null, null, null, 5] | ... s:drop() | --- | ... - -s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) - | --- - | ... -_ = s:format({{name='id'}, {name='data', type='array', is_nullable=true}}) - | --- - | ... -_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) - | --- - | ... -s:replace{1, box.NULL} - | --- - | - [1, null] - | ... -_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) - | --- - | - error: multikey index does not support nullable root field - | ... -s:replace{2, box.NULL} - | --- - | - [2, null] - | ... -s:drop() - | --- - | ... - -s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) - | --- - | ... -_ = s:format({{name='id'}, {name='data', type='array'}}) - | --- - | ... -_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) - | --- - | ... -s:replace{1, box.NULL} - | --- - | - error: 'Tuple field 2 type does not match one required by operation: expected array' - | ... -_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) - | --- - | ... -s:replace{2, box.NULL} - | --- - | - error: 'Tuple field 2 type does not match one required by operation: expected array' - | ... -s:replace{3, {}} - | --- - | - [3, []] - | ... -s:drop() - | --- - | ... diff --git a/test/engine/gh-5027-fields-nullability.test.lua b/test/engine/gh-5027-fields-nullability.test.lua index 960103d6c..664883d0d 100644 --- a/test/engine/gh-5027-fields-nullability.test.lua +++ b/test/engine/gh-5027-fields-nullability.test.lua @@ -3,35 +3,18 @@ test_run = require('test_run').new() s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) _ = s:create_index('i1', {parts={{1, 'unsigned'}}}) _ = s:create_index('i2', {parts={{5, 'unsigned', is_nullable=true}}}) -s:replace{1} -s:replace{1, box.NULL} -s:replace{1, box.NULL, box.NULL} -s:replace{1, box.NULL, box.NULL, box.NULL} +s:replace{1} -- ok +s:replace{1, box.NULL} -- ok +s:replace{1, box.NULL, box.NULL} -- ok +s:replace{1, box.NULL, box.NULL, box.NULL} -- ok s:drop() s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) _ = s:create_index('i1', {parts={{1, 'unsigned'}}}) _ = s:create_index('i2', {parts={{5, 'unsigned', is_nullable=false}}}) -s:replace{1} -s:replace{1, box.NULL} -s:replace{1, box.NULL, box.NULL} -s:replace{1, box.NULL, box.NULL, box.NULL} -s:replace{1, box.NULL, box.NULL, box.NULL, 5} -s:drop() - -s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) -_ = s:format({{name='id'}, {name='data', type='array', is_nullable=true}}) -_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) -s:replace{1, box.NULL} -_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) -s:replace{2, box.NULL} -s:drop() - -s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) -_ = s:format({{name='id'}, {name='data', type='array'}}) -_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) -s:replace{1, box.NULL} -_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) -s:replace{2, box.NULL} -s:replace{3, {}} +s:replace{1} -- error +s:replace{1, box.NULL} -- error +s:replace{1, box.NULL, box.NULL} -- error +s:replace{1, box.NULL, box.NULL, box.NULL} -- error +s:replace{1, box.NULL, box.NULL, box.NULL, 5} -- ok s:drop() diff --git a/test/engine/gh-5192-multikey-root-nullability.result b/test/engine/gh-5192-multikey-root-nullability.result new file mode 100644 index 000000000..dcbb5067f --- /dev/null +++ b/test/engine/gh-5192-multikey-root-nullability.result @@ -0,0 +1,62 @@ +-- test-run result file version 2 +test_run = require('test_run').new() + | --- + | ... + +s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) + | --- + | ... +_ = s:format({{name='id'}, {name='data', type='array', is_nullable=true}}) + | --- + | ... +_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) + | --- + | ... +s:replace{1, box.NULL} -- ok + | --- + | - [1, null] + | ... +_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) + | --- + | ... +s:replace{2, box.NULL} -- ok + | --- + | - [2, null] + | ... +_ = s:delete(2) + | --- + | ... +s:truncate() + | --- + | ... +s:drop() + | --- + | ... + +s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) + | --- + | ... +_ = s:format({{name='id'}, {name='data', type='array'}}) + | --- + | ... +_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) + | --- + | ... +s:replace{1, box.NULL} -- error + | --- + | - error: 'Tuple field 2 type does not match one required by operation: expected array' + | ... +_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) + | --- + | ... +s:replace{2, box.NULL} -- error + | --- + | - error: 'Tuple field 2 type does not match one required by operation: expected array' + | ... +s:replace{3, {}} -- ok + | --- + | - [3, []] + | ... +s:drop() + | --- + | ... diff --git a/test/engine/gh-5192-multikey-root-nullability.test.lua b/test/engine/gh-5192-multikey-root-nullability.test.lua new file mode 100644 index 000000000..5a8305bf9 --- /dev/null +++ b/test/engine/gh-5192-multikey-root-nullability.test.lua @@ -0,0 +1,20 @@ +test_run = require('test_run').new() + +s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) +_ = s:format({{name='id'}, {name='data', type='array', is_nullable=true}}) +_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) +s:replace{1, box.NULL} -- ok +_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) +s:replace{2, box.NULL} -- ok +_ = s:delete(2) +s:truncate() +s:drop() + +s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) +_ = s:format({{name='id'}, {name='data', type='array'}}) +_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) +s:replace{1, box.NULL} -- error +_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) +s:replace{2, box.NULL} -- error +s:replace{3, {}} -- ok +s:drop() \ No newline at end of file diff --git a/test/engine/json.result b/test/engine/json.result index 2175266fc..b8fd9a1b6 100644 --- a/test/engine/json.result +++ b/test/engine/json.result @@ -738,12 +738,11 @@ _ = s:create_index('sk', {parts = {{'[2][1].a', 'unsigned'}}}) ... s:insert{1, box.NULL} -- error --- -- error: 'Tuple field 2 type does not match one required by operation: expected array' +- error: Tuple field [2][1]["a"] required by space format is missing ... s:insert{2, {box.NULL}} -- error --- -- error: 'Tuple field [2][1] type does not match one required by operation: expected - map' +- error: Tuple field [2][1]["a"] required by space format is missing ... s:insert{3} -- error --- @@ -764,16 +763,15 @@ s:insert{6, {{a = 1}}} -- ok s.index.sk:alter{parts = {{'[2][1].a', 'unsigned', is_nullable = true}}} --- ... -s:insert{7, box.NULL} -- error +s:insert{7, box.NULL} -- ok --- -- error: 'Tuple field 2 type does not match one required by operation: expected array' +- [7, null] ... -s:insert{8, {box.NULL}} -- error +s:insert{8, {box.NULL}} -- ok --- -- error: 'Tuple field [2][1] type does not match one required by operation: expected - map' +- [8, [null]] ... --- Skipping nullable fields is okay though. +-- Skipping nullable fields is also okay. s:insert{9} -- ok --- - [9] @@ -792,7 +790,9 @@ s:insert{12, {{a = box.NULL}}} -- ok ... s.index.sk:select() --- -- - [9] +- - [7, null] + - [8, [null]] + - [9] - [10, []] - [11, [{'b': 1}]] - [12, [{'a': null}]] diff --git a/test/engine/json.test.lua b/test/engine/json.test.lua index 4771c3162..371bbad91 100644 --- a/test/engine/json.test.lua +++ b/test/engine/json.test.lua @@ -220,9 +220,9 @@ s:insert{4, {}} -- error s:insert{5, {{b = 1}}} -- error s:insert{6, {{a = 1}}} -- ok s.index.sk:alter{parts = {{'[2][1].a', 'unsigned', is_nullable = true}}} -s:insert{7, box.NULL} -- error -s:insert{8, {box.NULL}} -- error --- Skipping nullable fields is okay though. +s:insert{7, box.NULL} -- ok +s:insert{8, {box.NULL}} -- ok +-- Skipping nullable fields is also okay. s:insert{9} -- ok s:insert{10, {}} -- ok s:insert{11, {{b = 1}}} -- ok diff --git a/test/engine/multikey.result b/test/engine/multikey.result index 968be4cc3..ff72983ec 100644 --- a/test/engine/multikey.result +++ b/test/engine/multikey.result @@ -794,9 +794,9 @@ _ = s:create_index('pk') _ = s:create_index('sk', {parts = {{'[2][*]', 'unsigned'}}}) --- ... -s:insert{1, box.NULL} -- error +s:insert{1, box.NULL} -- ok --- -- error: 'Tuple field 2 type does not match one required by operation: expected array' +- [1, null] ... s:insert{2, {box.NULL}} -- error --- @@ -814,9 +814,9 @@ s:insert{4, {1}} -- ok s.index.sk:alter{parts = {{'[2][*]', 'unsigned', is_nullable = true}}} --- ... -s:insert{5, box.NULL} -- still error +s:insert{5, box.NULL} -- ok --- -- error: 'Tuple field 2 type does not match one required by operation: expected array' +- [5, null] ... s:insert{6, {box.NULL}} -- ok --- diff --git a/test/engine/multikey.test.lua b/test/engine/multikey.test.lua index ed7033494..653a9c9b1 100644 --- a/test/engine/multikey.test.lua +++ b/test/engine/multikey.test.lua @@ -214,12 +214,12 @@ s:drop() s = box.schema.space.create('test', {engine = engine}) _ = s:create_index('pk') _ = s:create_index('sk', {parts = {{'[2][*]', 'unsigned'}}}) -s:insert{1, box.NULL} -- error +s:insert{1, box.NULL} -- ok s:insert{2, {box.NULL}} -- error s:insert{3, {}} -- ok s:insert{4, {1}} -- ok s.index.sk:alter{parts = {{'[2][*]', 'unsigned', is_nullable = true}}} -s:insert{5, box.NULL} -- still error +s:insert{5, box.NULL} -- ok s:insert{6, {box.NULL}} -- ok s:insert{7, {}} -- ok s:insert{8, {2}} -- ok -- 2.17.1 From korablev at tarantool.org Wed Aug 5 15:29:21 2020 From: korablev at tarantool.org (Nikita Pettik) Date: Wed, 5 Aug 2020 12:29:21 +0000 Subject: [Tarantool-patches] [PATCH v2] tuple: drop extra restrictions for multikey index In-Reply-To: <20200805101251.30600-1-i.kosarev@tarantool.org> References: <20200805101251.30600-1-i.kosarev@tarantool.org> Message-ID: <20200805122921.GA15209@tarantool.org> On 05 Aug 13:12, Ilya Kosarev wrote: > @ChangeLog: > * Dropped restrictions on nullable multikey index root. All fields are > now nullable by default as it was before 2.2.1 (gh-5192). I'd underline the fact that problem was fixed in another (local code change) way. What is more, I guess we should document this behaviour: our current docs say nothing concerning nulls as multikey roots. > diff --git a/test/engine/gh-5027-fields-nullability.test.lua b/test/engine/gh-5027-fields-nullability.test.lua > index 960103d6c..664883d0d 100644 > --- a/test/engine/gh-5027-fields-nullability.test.lua > +++ b/test/engine/gh-5027-fields-nullability.test.lua > @@ -3,35 +3,18 @@ test_run = require('test_run').new() > s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) > _ = s:create_index('i1', {parts={{1, 'unsigned'}}}) > _ = s:create_index('i2', {parts={{5, 'unsigned', is_nullable=true}}}) > -s:replace{1} > -s:replace{1, box.NULL} > -s:replace{1, box.NULL, box.NULL} > -s:replace{1, box.NULL, box.NULL, box.NULL} > +s:replace{1} -- ok > +s:replace{1, box.NULL} -- ok > +s:replace{1, box.NULL, box.NULL} -- ok > +s:replace{1, box.NULL, box.NULL, box.NULL} -- ok I'd drop these -- ok comments since they only enlarge diff. > diff --git a/test/engine/gh-5192-multikey-root-nullability.test.lua b/test/engine/gh-5192-multikey-root-nullability.test.lua > new file mode 100644 > index 000000000..5a8305bf9 > --- /dev/null > +++ b/test/engine/gh-5192-multikey-root-nullability.test.lua > @@ -0,0 +1,20 @@ > +test_run = require('test_run').new() > + > +s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) > +_ = s:format({{name='id'}, {name='data', type='array', is_nullable=true}}) > +_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) > +s:replace{1, box.NULL} -- ok > +_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) > +s:replace{2, box.NULL} -- ok > +_ = s:delete(2) > +s:truncate() Why do you need to truncate space before drop? > +s:drop() > + > +s = box.schema.space.create('gh-5027', {engine=test_run:get_cfg('engine')}) > +_ = s:format({{name='id'}, {name='data', type='array'}}) > +_ = s:create_index('i1', {parts={{1, 'unsigned'}}}) > +s:replace{1, box.NULL} -- error > +_ = s:create_index('i2', {parts={{field=2, path='[*].key', type='string'}}}) > +s:replace{2, box.NULL} -- error > +s:replace{3, {}} -- ok > +s:drop() > \ No newline at end of file Nit: put pls newline at the end of file to avoid this warnings. > diff --git a/test/engine/json.test.lua b/test/engine/json.test.lua > index 4771c3162..371bbad91 100644 > --- a/test/engine/json.test.lua > +++ b/test/engine/json.test.lua > @@ -220,9 +220,9 @@ s:insert{4, {}} -- error > s:insert{5, {{b = 1}}} -- error > s:insert{6, {{a = 1}}} -- ok > s.index.sk:alter{parts = {{'[2][1].a', 'unsigned', is_nullable = true}}} > -s:insert{7, box.NULL} -- error > -s:insert{8, {box.NULL}} -- error > --- Skipping nullable fields is okay though. > +s:insert{7, box.NULL} -- ok > +s:insert{8, {box.NULL}} -- ok > +-- Skipping nullable fields is also okay. Is this change really required? :) > s:insert{9} -- ok > s:insert{10, {}} -- ok > s:insert{11, {{b = 1}}} -- ok From huston.mavr at gmail.com Wed Aug 5 20:08:13 2020 From: huston.mavr at gmail.com (Mavr Huston) Date: Wed, 5 Aug 2020 20:08:13 +0300 Subject: [Tarantool-patches] [PATCH] build: refactor static build process In-Reply-To: <20200727223734.cnxrdzik2cyt3ey4@tkn_work_nb> References: <20200622181649.10100-1-huston.mavr@gmail.com> <20200727223734.cnxrdzik2cyt3ey4@tkn_work_nb> Message-ID: Hi, thanks for the review! libicu installs as ExternalProject_Add too, its missed in commit message; Problem with curses and ncurses was on macOS and linux, because libcurses is an entire copy of libncurses, and tarantool links with system libcurses instead of libncurses installed as tarantool dependency, but module FindCurses.cmkae provides workaround for this problem - CURSES_NEED_NCURSES flag.- to use ncurses instead of curses. (i will fix this part at commit message) About disable-shred flag, used at libcurl building - we want to link only with static libraries, so we prevent creating unused .so. I've renamed static_build_no_deps_* jobs after review to static_build_cmake_* Also about such path tarantool-prefix/* - it's a cmake ExternalProject_Add() default path (i've also added comment at .travis.mk) Useless comments "Init macOS test env" deleted. > if (BUILD_STATIC) > - set(LIBZ_LIB_NAME libz.a) > + find_library(LIBZ_LIBRARY NAMES libz.a) > else() > - set(LIBZ_LIB_NAME z) > + find_library(LIBZ_LIBRARY NAMES z) > endif() > - find_library(LIBZ_LIBRARY NAMES ${LIBZ_LIB_NAME}) Here we simplified code, by deleting useless variable. I've added commentaries to cmake/compiler.cmake about libunwind on macOS and about ignoring flag -static-libstdc++ on macOS I've fixed static-build for using system compiler: gcc/g++ on linux and clang/clang++ on macOS I've refactored IF (NOT APPLE) condition to IF (APPLE) at static-build/CMakeLists.txt I've mentioned macOS dependencies at static-build/README.md xcode-tools and others, also I've added example with CMAKE_TARANTOOL_ARGS. Added commentaries about _EP_INSTALL_DIR at static-build/CMakeLists.txt Also deleted unused use_unix_sockets_iproto = True Also deleted curl-features.test.lua, because after rebase on master it fails, due to missing curl_version_info symbol at tarantool binary. This symbol lost after #807c7fa584f21ee955b2a14623d70f7510a3650d (build: update curl submodule to 7.71.1 version ) After pass the review I'll squash this changes to base commit and update commit message. Here is a diff of changes: ===================================== diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5ec5dd9b9..c9aef3dc7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -534,14 +534,14 @@ static_build: script: - ${GITLAB_MAKE} test_static_build -static_build_no_deps_linux: +static_build_cmake_linux: <<: *docker_test_definition script: - - ${GITLAB_MAKE} test_static_build_no_deps_linux + - ${GITLAB_MAKE} test_static_build_cmake_linux -static_build_no_deps_osx_15: +static_build_cmake_osx_15: stage: test tags: - osx_15 script: - - ${GITLAB_MAKE} test_static_build_no_deps_osx + - ${GITLAB_MAKE} test_static_build_cmake_osx diff --git a/.travis.mk b/.travis.mk index 64862348f..482672429 100644 --- a/.travis.mk +++ b/.travis.mk @@ -149,8 +149,8 @@ test_static_build: deps_debian_static CMAKE_EXTRA_PARAMS=-DBUILD_STATIC=ON make -f .travis.mk test_debian_no_deps # New static build - -test_static_build_no_deps_linux: +# builddir used in this target - is a default build path from cmake ExternalProject_Add() +test_static_build_cmake_linux: cd static-build && cmake . && make -j && ctest -V cd test && /usr/bin/python test-run.py --force \ --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build $(TEST_RUN_EXTRA_PARAMS) @@ -218,7 +218,6 @@ INIT_TEST_ENV_OSX=\ rm -rf /tmp/tnt test_osx_no_deps: build_osx - # Init macOS test env ${INIT_TEST_ENV_OSX}; \ cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) @@ -233,9 +232,9 @@ base_deps_osx: brew install --force ${STATIC_OSX_PKGS} || brew upgrade ${STATIC_OSX_PKGS} pip install --force-reinstall -r test-run/requirements.txt -test_static_build_no_deps_osx: base_deps_osx +# builddir used in this target - is a default build path from cmake ExternalProject_Add() +test_static_build_cmake_osx: base_deps_osx cd static-build && cmake . && make -j && ctest -V - # Init macOS test env ${INIT_TEST_ENV_OSX}; \ cd test && ./test-run.py --vardir /tmp/tnt \ --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build \ diff --git a/cmake/FindReadline.cmake b/cmake/FindReadline.cmake index 8422181d6..afe480679 100644 --- a/cmake/FindReadline.cmake +++ b/cmake/FindReadline.cmake @@ -14,6 +14,14 @@ if(BUILD_STATIC) if (NOT CURSES_INFO_LIBRARY) set(CURSES_INFO_LIBRARY "") endif() + + # From Modules/FindCurses.cmake: + # Set ``CURSES_NEED_NCURSES`` to ``TRUE`` before the + # ``find_package(Curses)`` call if NCurses functionality is required. + # This flag is set for linking with required library (installed + # via static-build/CMakeLists.txt). If this variable won't be set + # then tarantool binary links with system library curses which is an + # entire copy of ncurses set(CURSES_NEED_NCURSES TRUE) endif() diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake index 14f1e1186..db2ae6227 100644 --- a/cmake/compiler.cmake +++ b/cmake/compiler.cmake @@ -131,6 +131,8 @@ set(CMAKE_REQUIRED_INCLUDES "") if(BUILD_STATIC AND NOT TARGET_OS_DARWIN) set(UNWIND_LIB_NAME libunwind.a) else() + # libunwind can't be compiled on macOS. + # But there exists libunwind.dylib as a part of MacOSSDK set(UNWIND_LIB_NAME unwind) endif() find_library(UNWIND_LIBRARY PATH_SUFFIXES system NAMES ${UNWIND_LIB_NAME}) @@ -192,6 +194,9 @@ if (ENABLE_BACKTRACE) find_package_message(UNWIND_LIBRARIES "Found unwind" "${UNWIND_LIBRARIES}") endif() +# On macOS there is no '-static-libstdc++' flag and it's use will +# raise following error: +# error: argument unused during compilation: '-static-libstdc++' if(BUILD_STATIC AND NOT TARGET_OS_DARWIN) # Static linking for c++ routines add_compile_flags("C;CXX" "-static-libstdc++") diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt index 86582af0a..53ceb609c 100644 --- a/static-build/CMakeLists.txt +++ b/static-build/CMakeLists.txt @@ -9,11 +9,18 @@ set(NCURSES_VERSION 6.2) set(READLINE_VERSION 8.0) set(UNWIND_VERSION 1.3-rc1) -find_program(C_COMPILER gcc) -find_program(CXX_COMPILER g++) +if (APPLE) + find_program(C_COMPILER clang) + find_program(CXX_COMPILER clang++) +else() + find_program(C_COMPILER gcc) + find_program(CXX_COMPILER g++) +endif() set(CMAKE_C_COMPILER ${C_COMPILER}) set(CMAKE_CXX_COMPILER ${CXX_COMPILER}) +# Install all libraries required by tarantool at current build dir + # # OpenSSL # @@ -80,7 +87,18 @@ ExternalProject_Add(readline # # ICONV # -if (NOT APPLE) +if (APPLE) + ExternalProject_Add(iconv + URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz + CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER} + /configure + --prefix= + --disable-shared + --enable-static + --with-gnu-ld + STEP_TARGETS download + ) +else() # In linux iconv is embedded into glibc # So we find system header and copy it locally find_path(ICONV_INCLUDE_DIR iconv.h) @@ -101,20 +119,11 @@ if (NOT APPLE) add_custom_target(iconv DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix/include/iconv.h" ) + # This is a hack for further getting install directory of library + # by ExternalProject_Get_Property set_target_properties(iconv PROPERTIES _EP_INSTALL_DIR ${ICONV_INSTALL_PREFIX} ) -else() - ExternalProject_Add(iconv - URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz - CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER} - /configure - --prefix= - --disable-shared - --enable-static - --with-gnu-ld - STEP_TARGETS download - ) endif() # @@ -162,6 +171,8 @@ if (APPLE) endif() add_custom_target(unwind DEPENDS ${UNWIND_DEPENDENCIES}) + # This is a hack for further getting install directory of library + # by ExternalProject_Get_Property set_target_properties(unwind PROPERTIES _EP_INSTALL_DIR ${UNWIND_INSTALL_PREFIX} ) @@ -178,6 +189,8 @@ else() ) endif() +# Get install directories of builded libraries for building +# tarantool with custon CMAKE_PREFIX_PATH foreach(PROJ openssl icu zlib ncurses readline iconv unwind) ExternalProject_Get_Property(${PROJ} install_dir) set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH}:${install_dir}) @@ -197,16 +210,14 @@ ExternalProject_Add(tarantool -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH} -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=FALSE -DOPENSSL_USE_STATIC_LIBS=TRUE - -DCMAKE_BUILD_TYPE=Debug -DBUILD_STATIC=TRUE -DENABLE_DIST=TRUE -DENABLE_BACKTRACE=TRUE - -DPACKAGE:STRING=${PACKAGE_NAME} -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} ${CMAKE_TARANTOOL_ARGS} - BUILD_COMMAND ${CMAKE_MAKE_PROGRAM} -j STEP_TARGETS build + BUILD_COMMAND $(MAKE) ) enable_testing() diff --git a/static-build/README.md b/static-build/README.md index 29fe085c3..0019e963f 100644 --- a/static-build/README.md +++ b/static-build/README.md @@ -13,6 +13,24 @@ yum install -y \ python-msgpack python-yaml python-argparse python-six python-gevent ``` +MacOS: + +Before you start please install default Xcode Tools by Apple: + +``` +sudo xcode-select --install +sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer +``` + +Install brew using command from +[Homebrew repository instructions](https://github.com/Homebrew/inst) + +After that run next script: + +```bash + brew install autoconf automake libtool cmake file://$${PWD}/tools/brew_taps/tntpython2.rbs + pip install --force-reinstall -r test-run/requirements.txt +``` ### Usage @@ -21,3 +39,20 @@ cmake . make -j ctest -V ``` + +## Customize your build + +If you want to customise build, you need to set `CMAKE_TARANTOOL_ARGS` variable + +### Usage + +There is three types of `CMAKE_BUILD_TYPE`: +* Debug - default +* Release +* RelWithDebInfo + +And you want to build tarantool with RelWithDebInfo: + +```bash +cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo" . +``` diff --git a/static-build/test/static-build/curl-features.test.lua b/static-build/test/static-build/curl-features.test.lua deleted file mode 100755 index 57b1c4306..000000000 --- a/static-build/test/static-build/curl-features.test.lua +++ /dev/null @@ -1,67 +0,0 @@ -#!/usr/bin/env tarantool - -local tap = require('tap') -local ffi = require('ffi') -ffi.cdef([[ - struct curl_version_info_data { - int age; /* see description below */ - const char *version; /* human readable string */ - unsigned int version_num; /* numeric representation */ - const char *host; /* human readable string */ - int features; /* bitmask, see below */ - char *ssl_version; /* human readable string */ - long ssl_version_num; /* not used, always zero */ - const char *libz_version; /* human readable string */ - const char * const *protocols; /* protocols */ - - /* when 'age' is CURLVERSION_SECOND or higher, the members below exist */ - const char *ares; /* human readable string */ - int ares_num; /* number */ - - /* when 'age' is CURLVERSION_THIRD or higher, the members below exist */ - const char *libidn; /* human readable string */ - - /* when 'age' is CURLVERSION_FOURTH or higher (>= 7.16.1), the members - below exist */ - int iconv_ver_num; /* '_libiconv_version' if iconv support enabled */ - - const char *libssh_version; /* human readable string */ - - /* when 'age' is CURLVERSION_FIFTH or higher (>= 7.57.0), the members - below exist */ - unsigned int brotli_ver_num; /* Numeric Brotli version - (MAJOR << 24) | (MINOR << 12) | PATCH */ - const char *brotli_version; /* human readable string. */ - - /* when 'age' is CURLVERSION_SIXTH or higher (>= 7.66.0), the members - below exist */ - unsigned int nghttp2_ver_num; /* Numeric nghttp2 version - (MAJOR << 16) | (MINOR << 8) | PATCH */ - const char *nghttp2_version; /* human readable string. */ - - const char *quic_version; /* human readable quic (+ HTTP/3) library + - version or NULL */ - - /* when 'age' is CURLVERSION_SEVENTH or higher (>= 7.70.0), the members - below exist */ - const char *cainfo; /* the built-in default CURLOPT_CAINFO, might - be NULL */ - const char *capath; /* the built-in default CURLOPT_CAPATH, might - be NULL */ - }; - - struct curl_version_info_data *curl_version_info(int age); -]]) - -local info = ffi.C.curl_version_info(7) -local test = tap.test('curl-features') -test:plan(2) - -if test:ok(info.ssl_version ~= nil, 'Curl built with SSL support') then - test:diag('ssl_version: ' .. ffi.string(info.ssl_version)) -end -if test:ok(info.libz_version ~= nil, 'Curl built with LIBZ') then - test:diag('libz_version: ' .. ffi.string(info.libz_version)) -end - -os.exit(test:check() and 0 or 1) diff --git a/static-build/test/static-build/suite.ini b/static-build/test/static-build/suite.ini index 4da3d5d2f..92e349466 100644 --- a/static-build/test/static-build/suite.ini +++ b/static-build/test/static-build/suite.ini @@ -3,4 +3,3 @@ core = app description = Static build tests script = box.lua is_parallel = True -use_unix_sockets_iproto = True -------------- next part -------------- An HTML attachment was scrubbed... URL: From v.shpilevoy at tarantool.org Thu Aug 6 01:15:46 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 00:15:46 +0200 Subject: [Tarantool-patches] [PATCH vshard 1/1] storage: allow replica to boot before master In-Reply-To: References: <0963ccd10f91b249116916d5c1c91f5eef11e438.1596575031.git.v.shpilevoy@tarantool.org> Message-ID: Thanks for the review! > diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua > index c6a78fe..ed577f9 100644 > --- a/vshard/storage/init.lua > +++ b/vshard/storage/init.lua > @@ -2435,8 +2446,11 @@ local function storage_cfg(cfg, this_replica_uuid, is_reload) > ? ? ?local uri = luri.parse(this_replica.uri) > ? ? ?schema_upgrade(is_master, uri.login, uri.password) > > -? ? local old_trigger = box.space._bucket:on_replace()[1] > -? ? box.space._bucket:on_replace(bucket_generation_increment, old_trigger) > +? ? if is_master then > +? ? ? ? local old_trigger = M.bucket_on_replace > +? ? ? ? box.space._bucket:on_replace(bucket_generation_increment, old_trigger) > +? ? ? ? M.bucket_on_replace = bucket_generation_increment > +? ? end > > > It seems that the trigger is never removed. If a replica was a master some time ago, it still has the trigger. > Is it safe? Yes, it is. The trigger only increments a counter in M. And the counter is not used for anything out of the node. It should be just locally monotonic. So it is ok to increment it even on replica. After all this is how it worked before. But again, after this comment I had second thoughts, and decided to remove the trigger when switched to replica to be consistent. See v2 in a new mail thread. From v.shpilevoy at tarantool.org Thu Aug 6 01:15:47 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 00:15:47 +0200 Subject: [Tarantool-patches] [PATCH vshard 1/1] storage: allow replica to boot before master In-Reply-To: References: <0963ccd10f91b249116916d5c1c91f5eef11e438.1596575031.git.v.shpilevoy@tarantool.org> Message-ID: <326078fd-79f7-7052-37e1-cbbd118e114c@tarantool.org> Hi! Thanks for the review! > diff --git a/test/lua_libs/storage_template.lua b/test/lua_libs/storage_template.lua > index 29a753d..84e4180 100644 > --- a/test/lua_libs/storage_template.lua > +++ b/test/lua_libs/storage_template.lua > @@ -10,12 +12,63 @@ log = require('log') > ?if not cfg.shard_index then > ? ? ?cfg.shard_index = 'bucket_id' > ?end > +instance_uuid = util.name_to_uuid[NAME] > + > +-- > +-- Bootstrap the instance exactly like vshard does. But don't > +-- initialize any vshard-specific code. > +-- > +local function boot_like_vshard() > +? ? assert(type(box.cfg) == 'function') > +? ? for rs_uuid, rs in pairs(cfg.sharding) do > +? ? ? ? for replica_uuid, replica in pairs(rs.replicas) do > +? ? ? ? ? ? if replica_uuid == instance_uuid then > +? ? ? ? ? ? ? ? local box_cfg = {replication = {}} > +? ? ? ? ? ? ? ? box_cfg.instance_uuid = replica_uuid > +? ? ? ? ? ? ? ? box_cfg.replicaset_uuid = rs_uuid > +? ? ? ? ? ? ? ? box_cfg.listen = replica.uri > +? ? ? ? ? ? ? ? box_cfg.read_only = not replica.master > +? ? ? ? ? ? ? ? box_cfg.replication_connect_quorum = 0 > +? ? ? ? ? ? ? ? box_cfg.replication_timeout = 0.1 > +? ? ? ? ? ? ? ? for _, replica in pairs(rs.replicas) do > +? ? ? ? ? ? ? ? ? ? table.insert(box_cfg.replication, replica.uri) > +? ? ? ? ? ? ? ? end > +? ? ? ? ? ? ? ? box.cfg(box_cfg) > +? ? ? ? ? ? ? ? if not replica.master then > +? ? ? ? ? ? ? ? ? ? return > +? ? ? ? ? ? ? ? end > +? ? ? ? ? ? ? ? local uri = luri.parse(replica.uri) > +? ? ? ? ? ? ? ? box.schema.user.create(uri.login, { > +? ? ? ? ? ? ? ? ? ? password = uri.password, if_not_exists = true, > +? ? ? ? ? ? ? ? }) > +? ? ? ? ? ? ? ? box.schema.user.grant(uri.login, 'super') > +? ? ? ? ? ? ? ? return > +? ? ? ? ? ? end > +? ? ? ? end > +? ? end > +? ? assert(false) > > > Should there be some meaningful message? Nope. This is just a test, so if it will fail, I will fix it. Don't want to invent anything here. The code is unreachable, and assert(false) is telling that fine. If you mean the message which would help to locate the error - not needed as well. The assertion fail message will contain file name and line number. > +end > + > +local omit_cfg = false > +local i = 1 > +while arg[i] ~= nil do > +? ? local key = arg[i] > +? ? i = i + 1 > +? ? if key == 'boot_before_cfg' then > +? ? ? ? boot_like_vshard() > +? ? ? ? omit_cfg = true > +? ? end > +end > diff --git a/vshard/storage/reload_evolution.lua b/vshard/storage/reload_evolution.lua > index 5d09b11..f38af74 100644 > --- a/vshard/storage/reload_evolution.lua > +++ b/vshard/storage/reload_evolution.lua > @@ -17,6 +17,14 @@ migrations[#migrations + 1] = function(M) > ? ? ?-- Code to update Lua objects. > ?end > > +migrations[#migrations + 1] = function(M) > +? ? local bucket = box.space._bucket > +? ? if bucket then > +? ? ? ? assert(M.bucket_on_replace == nil) > +? ? ? ? M.bucket_on_replace = bucket:on_replace()[1] > +? ? end > +end > + > ?-- > ?-- Perform an update based on a version stored in `M` (internals). > ?-- @param M Old module internals which should be updated. > @@ -33,7 +41,7 @@ local function upgrade(M) > ? ? ? ? ?log.error(err_msg) > ? ? ? ? ?error(err_msg) > ? ? ?end > -? ? for i = start_version, #migrations? do > +? ? for i = start_version + 1, #migrations do > > > Is it already tested somewhere else (migrations I mean)? This change looks like a fix for some other problem. Good you noticed this. I thought this fix can't be separated, because reload evolution didn't have any meaningful migrations, and that couldn't be tested. But after your question I had second thoughts and it appears the bug does have a "test" already. I extracted it into a separate commit. Although the test is reverted by the second commit anyway. See v2 patchset in a new thread. From v.shpilevoy at tarantool.org Thu Aug 6 01:15:47 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 00:15:47 +0200 Subject: [Tarantool-patches] [PATCH v2 vshard 0/2] storage: allow replica to boot before master Message-ID: The patchset fixes a problem appearing in tarantool cartridge when replica is configured before master and fails with an ugly error without a serious reason for that. As a prerequisite the reload evolution subsystem is fixed, because it had a bug affecting the main commit. The bug didn't appear earlier, because the evolution was never used for anything so far. Changes in v2: - The reload evolution fix is moved into a separate commit; - Instance un-installs _bucket trigger when master is switched to replica. Branch: http://github.com/tarantool/vshard/tree/gerold103/gh-237-boot-replica-first Issue: https://github.com/tarantool/vshard/issues/237 Vladislav Shpilevoy (2): storage: fix reload applying migration twice storage: allow replica to boot before master test/lua_libs/storage_template.lua | 55 +++++++++++- test/misc/reconfigure.result | 33 +++++++ test/misc/reconfigure.test.lua | 9 ++ test/reload_evolution/storage.result | 10 +++ test/reload_evolution/storage.test.lua | 8 ++ test/router/boot_replica_first.result | 112 ++++++++++++++++++++++++ test/router/boot_replica_first.test.lua | 42 +++++++++ vshard/storage/init.lua | 21 ++++- vshard/storage/reload_evolution.lua | 10 ++- 9 files changed, 296 insertions(+), 4 deletions(-) create mode 100644 test/router/boot_replica_first.result create mode 100644 test/router/boot_replica_first.test.lua -- 2.21.1 (Apple Git-122.3) From v.shpilevoy at tarantool.org Thu Aug 6 01:15:48 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 00:15:48 +0200 Subject: [Tarantool-patches] [PATCH v2 vshard 1/2] storage: fix reload applying migration twice In-Reply-To: References: Message-ID: Reload evolution is a subsystem for storage hot code reload. When code is reloaded, it applies migration scripts one by one. But there was a bug - the last migration script was applied even when code version didn't really change. The bug didn't affect anything, since so far the reload evolution was not used for anything and had just one dummy migration script doing nothing. Now a first migration script is going to be added, so the bug is fixed before that. Needed for #237 --- test/reload_evolution/storage.result | 6 +++++- test/reload_evolution/storage.test.lua | 6 +++++- vshard/storage/reload_evolution.lua | 2 +- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/test/reload_evolution/storage.result b/test/reload_evolution/storage.result index ebf4fdc..dde0a1f 100644 --- a/test/reload_evolution/storage.result +++ b/test/reload_evolution/storage.result @@ -86,7 +86,11 @@ package.loaded['vshard.storage'] = nil vshard.storage = require("vshard.storage") --- ... -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil +-- Should be nil. Because there was a bug that reload always reapplied the last +-- migration callback. When it was fixed, the last callback wasn't called twice. +-- But since the callback was only one, now nothing is called, and nothing is +-- logged. +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil --- - true ... diff --git a/test/reload_evolution/storage.test.lua b/test/reload_evolution/storage.test.lua index 56c1693..7858112 100644 --- a/test/reload_evolution/storage.test.lua +++ b/test/reload_evolution/storage.test.lua @@ -35,7 +35,11 @@ box.space.test:insert({42, bucket_id_to_move}) package.path = original_package_path package.loaded['vshard.storage'] = nil vshard.storage = require("vshard.storage") -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil +-- Should be nil. Because there was a bug that reload always reapplied the last +-- migration callback. When it was fixed, the last callback wasn't called twice. +-- But since the callback was only one, now nothing is called, and nothing is +-- logged. +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil vshard.storage.internal.reload_version -- Make sure storage operates well. vshard.storage.bucket_force_drop(2000) diff --git a/vshard/storage/reload_evolution.lua b/vshard/storage/reload_evolution.lua index 5d09b11..1abc9e2 100644 --- a/vshard/storage/reload_evolution.lua +++ b/vshard/storage/reload_evolution.lua @@ -33,7 +33,7 @@ local function upgrade(M) log.error(err_msg) error(err_msg) end - for i = start_version, #migrations do + for i = start_version + 1, #migrations do local ok, err = pcall(migrations[i], M) if ok then log.info('vshard.storage.reload_evolution: upgraded to %d version', -- 2.21.1 (Apple Git-122.3) From v.shpilevoy at tarantool.org Thu Aug 6 01:15:49 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 00:15:49 +0200 Subject: [Tarantool-patches] [PATCH v2 vshard 2/2] storage: allow replica to boot before master In-Reply-To: References: Message-ID: <0cd777f3990939b32d96bf964dea55e8073e65bb.1596665591.git.v.shpilevoy@tarantool.org> Before the patch a replica couldn't call vshard.storage.cfg() before the master is bootstrapped. That could lead to an exception in the middle of vshard.storage.cfg(). It sounds not so bad and even looks unreachable, because replica bootstrap anyway is blocked in box.cfg() until a master node is started, otherwise the replica instance is terminated. But the box.cfg bootstrap != vshard bootstrap. It could be that both replica and master nodes are already bootstrapped and working, but vshard.storage.cfg() wasn't called yet. In that case vshard.storage.cfg() on the replica wouldn't block in box.cfg(), and would happily fail a few lines later in vshard.storage.cfg() at attempt to touch not existing box.space._bucket. That was the case for cartridge. The framework configures instances in its own way before vshard.storage.cfg(). Then it calls vshard.storage.cfg() on them in arbitrary order, and sometimes replicas were configured earlier than the master. The fail is fixed by simply skipping the _bucket's trigger installation on the replica node. Because so far it is not needed here for anything. The trigger's sole purpose is to increment bucket generation used only for DML on _bucket on the master node. Now vshard.storage.cfg() on replica is able to finish before master does the same. However the replica still won't be able to handle vshard.storage.* requests until receives vshard schema from master. Closes #237 --- test/lua_libs/storage_template.lua | 55 +++++++++++- test/misc/reconfigure.result | 33 +++++++ test/misc/reconfigure.test.lua | 9 ++ test/reload_evolution/storage.result | 16 ++-- test/reload_evolution/storage.test.lua | 14 +-- test/router/boot_replica_first.result | 112 ++++++++++++++++++++++++ test/router/boot_replica_first.test.lua | 42 +++++++++ vshard/storage/init.lua | 21 ++++- vshard/storage/reload_evolution.lua | 8 ++ 9 files changed, 297 insertions(+), 13 deletions(-) create mode 100644 test/router/boot_replica_first.result create mode 100644 test/router/boot_replica_first.test.lua diff --git a/test/lua_libs/storage_template.lua b/test/lua_libs/storage_template.lua index 29a753d..84e4180 100644 --- a/test/lua_libs/storage_template.lua +++ b/test/lua_libs/storage_template.lua @@ -1,5 +1,7 @@ #!/usr/bin/env tarantool +local luri = require('uri') + NAME = require('fio').basename(arg[0], '.lua') fiber = require('fiber') test_run = require('test_run').new() @@ -10,12 +12,63 @@ log = require('log') if not cfg.shard_index then cfg.shard_index = 'bucket_id' end +instance_uuid = util.name_to_uuid[NAME] + +-- +-- Bootstrap the instance exactly like vshard does. But don't +-- initialize any vshard-specific code. +-- +local function boot_like_vshard() + assert(type(box.cfg) == 'function') + for rs_uuid, rs in pairs(cfg.sharding) do + for replica_uuid, replica in pairs(rs.replicas) do + if replica_uuid == instance_uuid then + local box_cfg = {replication = {}} + box_cfg.instance_uuid = replica_uuid + box_cfg.replicaset_uuid = rs_uuid + box_cfg.listen = replica.uri + box_cfg.read_only = not replica.master + box_cfg.replication_connect_quorum = 0 + box_cfg.replication_timeout = 0.1 + for _, replica in pairs(rs.replicas) do + table.insert(box_cfg.replication, replica.uri) + end + box.cfg(box_cfg) + if not replica.master then + return + end + local uri = luri.parse(replica.uri) + box.schema.user.create(uri.login, { + password = uri.password, if_not_exists = true, + }) + box.schema.user.grant(uri.login, 'super') + return + end + end + end + assert(false) +end + +local omit_cfg = false +local i = 1 +while arg[i] ~= nil do + local key = arg[i] + i = i + 1 + if key == 'boot_before_cfg' then + boot_like_vshard() + omit_cfg = true + end +end vshard = require('vshard') echo_count = 0 cfg.replication_connect_timeout = 3 cfg.replication_timeout = 0.1 -vshard.storage.cfg(cfg, util.name_to_uuid[NAME]) + +if not omit_cfg then + vshard.storage.cfg(cfg, instance_uuid) +end + function bootstrap_storage(engine) box.once("testapp:schema:1", function() if rawget(_G, 'CHANGE_SPACE_IDS') then diff --git a/test/misc/reconfigure.result b/test/misc/reconfigure.result index be58eca..168be5d 100644 --- a/test/misc/reconfigure.result +++ b/test/misc/reconfigure.result @@ -277,6 +277,14 @@ box.cfg.replication --- - - storage:storage at 127.0.0.1:3301 ... +box.cfg.read_only +--- +- false +... +#box.space._bucket:on_replace() +--- +- 1 +... _ = test_run:switch('storage_2_a') --- ... @@ -303,6 +311,15 @@ box.cfg.replication - - storage:storage at 127.0.0.1:3303 - storage:storage at 127.0.0.1:3304 ... +box.cfg.read_only +--- +- true +... +-- Should be zero on the slave node. Even though earlier the node was a master. +#box.space._bucket:on_replace() +--- +- 0 +... _ = test_run:switch('storage_2_b') --- ... @@ -329,6 +346,14 @@ box.cfg.replication - - storage:storage at 127.0.0.1:3303 - storage:storage at 127.0.0.1:3304 ... +box.cfg.read_only +--- +- false +... +#box.space._bucket:on_replace() +--- +- 1 +... _ = test_run:switch('storage_3_a') --- ... @@ -354,6 +379,14 @@ box.cfg.replication --- - - storage:storage at 127.0.0.1:3306 ... +box.cfg.read_only +--- +- false +... +#box.space._bucket:on_replace() +--- +- 1 +... _ = test_run:switch('router_1') --- ... diff --git a/test/misc/reconfigure.test.lua b/test/misc/reconfigure.test.lua index 1b894c8..e891010 100644 --- a/test/misc/reconfigure.test.lua +++ b/test/misc/reconfigure.test.lua @@ -111,6 +111,8 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end table.sort(uris) uris box.cfg.replication +box.cfg.read_only +#box.space._bucket:on_replace() _ = test_run:switch('storage_2_a') info = vshard.storage.info() @@ -119,6 +121,9 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end table.sort(uris) uris box.cfg.replication +box.cfg.read_only +-- Should be zero on the slave node. Even though earlier the node was a master. +#box.space._bucket:on_replace() _ = test_run:switch('storage_2_b') info = vshard.storage.info() @@ -127,6 +132,8 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end table.sort(uris) uris box.cfg.replication +box.cfg.read_only +#box.space._bucket:on_replace() _ = test_run:switch('storage_3_a') info = vshard.storage.info() @@ -135,6 +142,8 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end table.sort(uris) uris box.cfg.replication +box.cfg.read_only +#box.space._bucket:on_replace() _ = test_run:switch('router_1') info = vshard.router.info() diff --git a/test/reload_evolution/storage.result b/test/reload_evolution/storage.result index dde0a1f..4652c4f 100644 --- a/test/reload_evolution/storage.result +++ b/test/reload_evolution/storage.result @@ -86,16 +86,22 @@ package.loaded['vshard.storage'] = nil vshard.storage = require("vshard.storage") --- ... --- Should be nil. Because there was a bug that reload always reapplied the last --- migration callback. When it was fixed, the last callback wasn't called twice. --- But since the callback was only one, now nothing is called, and nothing is --- logged. -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil --- - true ... vshard.storage.internal.reload_version --- +- 2 +... +-- +-- gh-237: should be only one trigger. During gh-237 the trigger installation +-- became conditional and therefore required to remember the current trigger +-- somewhere. When reloaded from the old version, the trigger needed to be +-- fetched from _bucket:on_replace(). +-- +#box.space._bucket:on_replace() +--- - 1 ... -- Make sure storage operates well. diff --git a/test/reload_evolution/storage.test.lua b/test/reload_evolution/storage.test.lua index 7858112..06f7117 100644 --- a/test/reload_evolution/storage.test.lua +++ b/test/reload_evolution/storage.test.lua @@ -35,12 +35,16 @@ box.space.test:insert({42, bucket_id_to_move}) package.path = original_package_path package.loaded['vshard.storage'] = nil vshard.storage = require("vshard.storage") --- Should be nil. Because there was a bug that reload always reapplied the last --- migration callback. When it was fixed, the last callback wasn't called twice. --- But since the callback was only one, now nothing is called, and nothing is --- logged. -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil vshard.storage.internal.reload_version +-- +-- gh-237: should be only one trigger. During gh-237 the trigger installation +-- became conditional and therefore required to remember the current trigger +-- somewhere. When reloaded from the old version, the trigger needed to be +-- fetched from _bucket:on_replace(). +-- +#box.space._bucket:on_replace() + -- Make sure storage operates well. vshard.storage.bucket_force_drop(2000) vshard.storage.bucket_force_create(2000) diff --git a/test/router/boot_replica_first.result b/test/router/boot_replica_first.result new file mode 100644 index 0000000..1705230 --- /dev/null +++ b/test/router/boot_replica_first.result @@ -0,0 +1,112 @@ +-- test-run result file version 2 +test_run = require('test_run').new() + | --- + | ... +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } + | --- + | ... +test_run:create_cluster(REPLICASET_1, 'router', {args = 'boot_before_cfg'}) + | --- + | ... +util = require('util') + | --- + | ... +util.wait_master(test_run, REPLICASET_1, 'box_1_a') + | --- + | ... +_ = test_run:cmd("create server router with script='router/router_2.lua'") + | --- + | ... +_ = test_run:cmd("start server router") + | --- + | ... + +-- +-- gh-237: replica should be able to boot before master. Before the issue was +-- fixed, replica always tried to install a trigger on _bucket space even when +-- it was not created on a master yet - that lead to an exception in +-- storage.cfg. Now it should not install the trigger at all, because anyway it +-- is not needed on replica for anything. +-- + +test_run:switch('box_1_b') + | --- + | - true + | ... +vshard.storage.cfg(cfg, instance_uuid) + | --- + | ... +-- _bucket is not created yet. Will fail. +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) + | --- + | - attempt to index field '_bucket' (a nil value) + | ... + +test_run:switch('default') + | --- + | - true + | ... +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') + | --- + | ... + +test_run:switch('box_1_a') + | --- + | - true + | ... +vshard.storage.cfg(cfg, instance_uuid) + | --- + | ... + +test_run:switch('box_1_b') + | --- + | - true + | ... +test_run:wait_lsn('box_1_b', 'box_1_a') + | --- + | ... +-- Fails, but gracefully. Because the bucket is not found here. +vshard.storage.call(1, 'read', 'echo', {100}) + | --- + | - null + | - bucket_id: 1 + | reason: Not found + | code: 1 + | type: ShardingError + | message: 'Cannot perform action with bucket 1, reason: Not found' + | name: WRONG_BUCKET + | ... +-- Should not have triggers. +#box.space._bucket:on_replace() + | --- + | - 0 + | ... + +test_run:switch('router') + | --- + | - true + | ... +vshard.router.bootstrap() + | --- + | - true + | ... +vshard.router.callro(1, 'echo', {100}) + | --- + | - 100 + | ... + +test_run:switch("default") + | --- + | - true + | ... +test_run:cmd('stop server router') + | --- + | - true + | ... +test_run:cmd('delete server router') + | --- + | - true + | ... +test_run:drop_cluster(REPLICASET_1) + | --- + | ... diff --git a/test/router/boot_replica_first.test.lua b/test/router/boot_replica_first.test.lua new file mode 100644 index 0000000..7b1b3fd --- /dev/null +++ b/test/router/boot_replica_first.test.lua @@ -0,0 +1,42 @@ +test_run = require('test_run').new() +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } +test_run:create_cluster(REPLICASET_1, 'router', {args = 'boot_before_cfg'}) +util = require('util') +util.wait_master(test_run, REPLICASET_1, 'box_1_a') +_ = test_run:cmd("create server router with script='router/router_2.lua'") +_ = test_run:cmd("start server router") + +-- +-- gh-237: replica should be able to boot before master. Before the issue was +-- fixed, replica always tried to install a trigger on _bucket space even when +-- it was not created on a master yet - that lead to an exception in +-- storage.cfg. Now it should not install the trigger at all, because anyway it +-- is not needed on replica for anything. +-- + +test_run:switch('box_1_b') +vshard.storage.cfg(cfg, instance_uuid) +-- _bucket is not created yet. Will fail. +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) + +test_run:switch('default') +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') + +test_run:switch('box_1_a') +vshard.storage.cfg(cfg, instance_uuid) + +test_run:switch('box_1_b') +test_run:wait_lsn('box_1_b', 'box_1_a') +-- Fails, but gracefully. Because the bucket is not found here. +vshard.storage.call(1, 'read', 'echo', {100}) +-- Should not have triggers. +#box.space._bucket:on_replace() + +test_run:switch('router') +vshard.router.bootstrap() +vshard.router.callro(1, 'echo', {100}) + +test_run:switch("default") +test_run:cmd('stop server router') +test_run:cmd('delete server router') +test_run:drop_cluster(REPLICASET_1) diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua index c6a78fe..5464824 100644 --- a/vshard/storage/init.lua +++ b/vshard/storage/init.lua @@ -94,6 +94,17 @@ if not M then -- detect that _bucket was not changed between yields. -- bucket_generation = 0, + -- + -- Reference to the function used as on_replace trigger on + -- _bucket space. It is used to replace the trigger with + -- a new function when reload happens. It is kept + -- explicitly because the old function is deleted on + -- reload from the global namespace. On the other hand, it + -- is still stored in _bucket:on_replace() somewhere, but + -- it is not known where. The only 100% way to be able to + -- replace the old function is to keep its reference. + -- + bucket_on_replace = nil, ------------------- Garbage collection ------------------- -- Fiber to remove garbage buckets data. @@ -2435,8 +2446,14 @@ local function storage_cfg(cfg, this_replica_uuid, is_reload) local uri = luri.parse(this_replica.uri) schema_upgrade(is_master, uri.login, uri.password) - local old_trigger = box.space._bucket:on_replace()[1] - box.space._bucket:on_replace(bucket_generation_increment, old_trigger) + if M.bucket_on_replace then + box.space._bucket:on_replace(nil, M.bucket_on_replace) + M.bucket_on_replace = nil + end + if is_master then + box.space._bucket:on_replace(bucket_generation_increment) + M.bucket_on_replace = bucket_generation_increment + end lreplicaset.rebind_replicasets(new_replicasets, M.replicasets) lreplicaset.outdate_replicasets(M.replicasets) diff --git a/vshard/storage/reload_evolution.lua b/vshard/storage/reload_evolution.lua index 1abc9e2..f38af74 100644 --- a/vshard/storage/reload_evolution.lua +++ b/vshard/storage/reload_evolution.lua @@ -17,6 +17,14 @@ migrations[#migrations + 1] = function(M) -- Code to update Lua objects. end +migrations[#migrations + 1] = function(M) + local bucket = box.space._bucket + if bucket then + assert(M.bucket_on_replace == nil) + M.bucket_on_replace = bucket:on_replace()[1] + end +end + -- -- Perform an update based on a version stored in `M` (internals). -- @param M Old module internals which should be updated. -- 2.21.1 (Apple Git-122.3) From olegrok at tarantool.org Thu Aug 6 09:40:02 2020 From: olegrok at tarantool.org (Oleg Babin) Date: Thu, 6 Aug 2020 09:40:02 +0300 Subject: [Tarantool-patches] [PATCH v2 vshard 1/2] storage: fix reload applying migration twice In-Reply-To: References: Message-ID: <6179c32d-0970-db1b-09a4-178d8ab102b0@tarantool.org> Hi! Thanks for your patch. LGTM. On 06/08/2020 01:15, Vladislav Shpilevoy wrote: > Reload evolution is a subsystem for storage hot code reload. When > code is reloaded, it applies migration scripts one by one. But > there was a bug - the last migration script was applied even when > code version didn't really change. > > The bug didn't affect anything, since so far the reload evolution > was not used for anything and had just one dummy migration script > doing nothing. > > Now a first migration script is going to be added, so the bug is > fixed before that. > > Needed for #237 > --- > test/reload_evolution/storage.result | 6 +++++- > test/reload_evolution/storage.test.lua | 6 +++++- > vshard/storage/reload_evolution.lua | 2 +- > 3 files changed, 11 insertions(+), 3 deletions(-) > > diff --git a/test/reload_evolution/storage.result b/test/reload_evolution/storage.result > index ebf4fdc..dde0a1f 100644 > --- a/test/reload_evolution/storage.result > +++ b/test/reload_evolution/storage.result > @@ -86,7 +86,11 @@ package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > --- > ... > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil > +-- Should be nil. Because there was a bug that reload always reapplied the last > +-- migration callback. When it was fixed, the last callback wasn't called twice. > +-- But since the callback was only one, now nothing is called, and nothing is > +-- logged. > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil > --- > - true > ... > diff --git a/test/reload_evolution/storage.test.lua b/test/reload_evolution/storage.test.lua > index 56c1693..7858112 100644 > --- a/test/reload_evolution/storage.test.lua > +++ b/test/reload_evolution/storage.test.lua > @@ -35,7 +35,11 @@ box.space.test:insert({42, bucket_id_to_move}) > package.path = original_package_path > package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil > +-- Should be nil. Because there was a bug that reload always reapplied the last > +-- migration callback. When it was fixed, the last callback wasn't called twice. > +-- But since the callback was only one, now nothing is called, and nothing is > +-- logged. > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil > vshard.storage.internal.reload_version > -- Make sure storage operates well. > vshard.storage.bucket_force_drop(2000) > diff --git a/vshard/storage/reload_evolution.lua b/vshard/storage/reload_evolution.lua > index 5d09b11..1abc9e2 100644 > --- a/vshard/storage/reload_evolution.lua > +++ b/vshard/storage/reload_evolution.lua > @@ -33,7 +33,7 @@ local function upgrade(M) > log.error(err_msg) > error(err_msg) > end > - for i = start_version, #migrations do > + for i = start_version + 1, #migrations do > local ok, err = pcall(migrations[i], M) > if ok then > log.info('vshard.storage.reload_evolution: upgraded to %d version', From olegrok at tarantool.org Thu Aug 6 09:40:12 2020 From: olegrok at tarantool.org (Oleg Babin) Date: Thu, 6 Aug 2020 09:40:12 +0300 Subject: [Tarantool-patches] [PATCH v2 vshard 2/2] storage: allow replica to boot before master In-Reply-To: <0cd777f3990939b32d96bf964dea55e8073e65bb.1596665591.git.v.shpilevoy@tarantool.org> References: <0cd777f3990939b32d96bf964dea55e8073e65bb.1596665591.git.v.shpilevoy@tarantool.org> Message-ID: <894ab65e-64f2-e593-4f58-1c239ce3994f@tarantool.org> Thanks for your patch. LGTM. On 06/08/2020 01:15, Vladislav Shpilevoy wrote: > Before the patch a replica couldn't call vshard.storage.cfg() > before the master is bootstrapped. That could lead to an exception > in the middle of vshard.storage.cfg(). > > It sounds not so bad and even looks unreachable, because replica > bootstrap anyway is blocked in box.cfg() until a master node is > started, otherwise the replica instance is terminated. > But the box.cfg bootstrap != vshard bootstrap. > > It could be that both replica and master nodes are already > bootstrapped and working, but vshard.storage.cfg() wasn't called > yet. In that case vshard.storage.cfg() on the replica wouldn't > block in box.cfg(), and would happily fail a few lines later in > vshard.storage.cfg() at attempt to touch not existing > box.space._bucket. > > That was the case for cartridge. The framework configures > instances in its own way before vshard.storage.cfg(). Then it > calls vshard.storage.cfg() on them in arbitrary order, and > sometimes replicas were configured earlier than the master. > > The fail is fixed by simply skipping the _bucket's trigger > installation on the replica node. Because so far it is not needed > here for anything. The trigger's sole purpose is to increment > bucket generation used only for DML on _bucket on the master node. > > Now vshard.storage.cfg() on replica is able to finish before > master does the same. However the replica still won't be able to > handle vshard.storage.* requests until receives vshard schema > from master. > > Closes #237 > --- > test/lua_libs/storage_template.lua | 55 +++++++++++- > test/misc/reconfigure.result | 33 +++++++ > test/misc/reconfigure.test.lua | 9 ++ > test/reload_evolution/storage.result | 16 ++-- > test/reload_evolution/storage.test.lua | 14 +-- > test/router/boot_replica_first.result | 112 ++++++++++++++++++++++++ > test/router/boot_replica_first.test.lua | 42 +++++++++ > vshard/storage/init.lua | 21 ++++- > vshard/storage/reload_evolution.lua | 8 ++ > 9 files changed, 297 insertions(+), 13 deletions(-) > create mode 100644 test/router/boot_replica_first.result > create mode 100644 test/router/boot_replica_first.test.lua > > diff --git a/test/lua_libs/storage_template.lua b/test/lua_libs/storage_template.lua > index 29a753d..84e4180 100644 > --- a/test/lua_libs/storage_template.lua > +++ b/test/lua_libs/storage_template.lua > @@ -1,5 +1,7 @@ > #!/usr/bin/env tarantool > > +local luri = require('uri') > + > NAME = require('fio').basename(arg[0], '.lua') > fiber = require('fiber') > test_run = require('test_run').new() > @@ -10,12 +12,63 @@ log = require('log') > if not cfg.shard_index then > cfg.shard_index = 'bucket_id' > end > +instance_uuid = util.name_to_uuid[NAME] > + > +-- > +-- Bootstrap the instance exactly like vshard does. But don't > +-- initialize any vshard-specific code. > +-- > +local function boot_like_vshard() > + assert(type(box.cfg) == 'function') > + for rs_uuid, rs in pairs(cfg.sharding) do > + for replica_uuid, replica in pairs(rs.replicas) do > + if replica_uuid == instance_uuid then > + local box_cfg = {replication = {}} > + box_cfg.instance_uuid = replica_uuid > + box_cfg.replicaset_uuid = rs_uuid > + box_cfg.listen = replica.uri > + box_cfg.read_only = not replica.master > + box_cfg.replication_connect_quorum = 0 > + box_cfg.replication_timeout = 0.1 > + for _, replica in pairs(rs.replicas) do > + table.insert(box_cfg.replication, replica.uri) > + end > + box.cfg(box_cfg) > + if not replica.master then > + return > + end > + local uri = luri.parse(replica.uri) > + box.schema.user.create(uri.login, { > + password = uri.password, if_not_exists = true, > + }) > + box.schema.user.grant(uri.login, 'super') > + return > + end > + end > + end > + assert(false) > +end > + > +local omit_cfg = false > +local i = 1 > +while arg[i] ~= nil do > + local key = arg[i] > + i = i + 1 > + if key == 'boot_before_cfg' then > + boot_like_vshard() > + omit_cfg = true > + end > +end > > vshard = require('vshard') > echo_count = 0 > cfg.replication_connect_timeout = 3 > cfg.replication_timeout = 0.1 > -vshard.storage.cfg(cfg, util.name_to_uuid[NAME]) > + > +if not omit_cfg then > + vshard.storage.cfg(cfg, instance_uuid) > +end > + > function bootstrap_storage(engine) > box.once("testapp:schema:1", function() > if rawget(_G, 'CHANGE_SPACE_IDS') then > diff --git a/test/misc/reconfigure.result b/test/misc/reconfigure.result > index be58eca..168be5d 100644 > --- a/test/misc/reconfigure.result > +++ b/test/misc/reconfigure.result > @@ -277,6 +277,14 @@ box.cfg.replication > --- > - -storage:storage at 127.0.0.1:3301 > ... > +box.cfg.read_only > +--- > +- false > +... > +#box.space._bucket:on_replace() > +--- > +- 1 > +... > _ = test_run:switch('storage_2_a') > --- > ... > @@ -303,6 +311,15 @@ box.cfg.replication > - -storage:storage at 127.0.0.1:3303 > -storage:storage at 127.0.0.1:3304 > ... > +box.cfg.read_only > +--- > +- true > +... > +-- Should be zero on the slave node. Even though earlier the node was a master. > +#box.space._bucket:on_replace() > +--- > +- 0 > +... > _ = test_run:switch('storage_2_b') > --- > ... > @@ -329,6 +346,14 @@ box.cfg.replication > - -storage:storage at 127.0.0.1:3303 > -storage:storage at 127.0.0.1:3304 > ... > +box.cfg.read_only > +--- > +- false > +... > +#box.space._bucket:on_replace() > +--- > +- 1 > +... > _ = test_run:switch('storage_3_a') > --- > ... > @@ -354,6 +379,14 @@ box.cfg.replication > --- > - -storage:storage at 127.0.0.1:3306 > ... > +box.cfg.read_only > +--- > +- false > +... > +#box.space._bucket:on_replace() > +--- > +- 1 > +... > _ = test_run:switch('router_1') > --- > ... > diff --git a/test/misc/reconfigure.test.lua b/test/misc/reconfigure.test.lua > index 1b894c8..e891010 100644 > --- a/test/misc/reconfigure.test.lua > +++ b/test/misc/reconfigure.test.lua > @@ -111,6 +111,8 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +#box.space._bucket:on_replace() > > _ = test_run:switch('storage_2_a') > info = vshard.storage.info() > @@ -119,6 +121,9 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +-- Should be zero on the slave node. Even though earlier the node was a master. > +#box.space._bucket:on_replace() > > _ = test_run:switch('storage_2_b') > info = vshard.storage.info() > @@ -127,6 +132,8 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +#box.space._bucket:on_replace() > > _ = test_run:switch('storage_3_a') > info = vshard.storage.info() > @@ -135,6 +142,8 @@ for k,v in pairs(info.replicasets) do table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +#box.space._bucket:on_replace() > > _ = test_run:switch('router_1') > info = vshard.router.info() > diff --git a/test/reload_evolution/storage.result b/test/reload_evolution/storage.result > index dde0a1f..4652c4f 100644 > --- a/test/reload_evolution/storage.result > +++ b/test/reload_evolution/storage.result > @@ -86,16 +86,22 @@ package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > --- > ... > --- Should be nil. Because there was a bug that reload always reapplied the last > --- migration callback. When it was fixed, the last callback wasn't called twice. > --- But since the callback was only one, now nothing is called, and nothing is > --- logged. > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil > --- > - true > ... > vshard.storage.internal.reload_version > --- > +- 2 > +... > +-- > +-- gh-237: should be only one trigger. During gh-237 the trigger installation > +-- became conditional and therefore required to remember the current trigger > +-- somewhere. When reloaded from the old version, the trigger needed to be > +-- fetched from _bucket:on_replace(). > +-- > +#box.space._bucket:on_replace() > +--- > - 1 > ... > -- Make sure storage operates well. > diff --git a/test/reload_evolution/storage.test.lua b/test/reload_evolution/storage.test.lua > index 7858112..06f7117 100644 > --- a/test/reload_evolution/storage.test.lua > +++ b/test/reload_evolution/storage.test.lua > @@ -35,12 +35,16 @@ box.space.test:insert({42, bucket_id_to_move}) > package.path = original_package_path > package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > --- Should be nil. Because there was a bug that reload always reapplied the last > --- migration callback. When it was fixed, the last callback wasn't called twice. > --- But since the callback was only one, now nothing is called, and nothing is > --- logged. > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') == nil > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: upgraded to') ~= nil > vshard.storage.internal.reload_version > +-- > +-- gh-237: should be only one trigger. During gh-237 the trigger installation > +-- became conditional and therefore required to remember the current trigger > +-- somewhere. When reloaded from the old version, the trigger needed to be > +-- fetched from _bucket:on_replace(). > +-- > +#box.space._bucket:on_replace() > + > -- Make sure storage operates well. > vshard.storage.bucket_force_drop(2000) > vshard.storage.bucket_force_create(2000) > diff --git a/test/router/boot_replica_first.result b/test/router/boot_replica_first.result > new file mode 100644 > index 0000000..1705230 > --- /dev/null > +++ b/test/router/boot_replica_first.result > @@ -0,0 +1,112 @@ > +-- test-run result file version 2 > +test_run = require('test_run').new() > + | --- > + | ... > +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } > + | --- > + | ... > +test_run:create_cluster(REPLICASET_1, 'router', {args = 'boot_before_cfg'}) > + | --- > + | ... > +util = require('util') > + | --- > + | ... > +util.wait_master(test_run, REPLICASET_1, 'box_1_a') > + | --- > + | ... > +_ = test_run:cmd("create server router with script='router/router_2.lua'") > + | --- > + | ... > +_ = test_run:cmd("start server router") > + | --- > + | ... > + > +-- > +-- gh-237: replica should be able to boot before master. Before the issue was > +-- fixed, replica always tried to install a trigger on _bucket space even when > +-- it was not created on a master yet - that lead to an exception in > +-- storage.cfg. Now it should not install the trigger at all, because anyway it > +-- is not needed on replica for anything. > +-- > + > +test_run:switch('box_1_b') > + | --- > + | - true > + | ... > +vshard.storage.cfg(cfg, instance_uuid) > + | --- > + | ... > +-- _bucket is not created yet. Will fail. > +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) > + | --- > + | - attempt to index field '_bucket' (a nil value) > + | ... > + > +test_run:switch('default') > + | --- > + | - true > + | ... > +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') > + | --- > + | ... > + > +test_run:switch('box_1_a') > + | --- > + | - true > + | ... > +vshard.storage.cfg(cfg, instance_uuid) > + | --- > + | ... > + > +test_run:switch('box_1_b') > + | --- > + | - true > + | ... > +test_run:wait_lsn('box_1_b', 'box_1_a') > + | --- > + | ... > +-- Fails, but gracefully. Because the bucket is not found here. > +vshard.storage.call(1, 'read', 'echo', {100}) > + | --- > + | - null > + | - bucket_id: 1 > + | reason: Not found > + | code: 1 > + | type: ShardingError > + | message: 'Cannot perform action with bucket 1, reason: Not found' > + | name: WRONG_BUCKET > + | ... > +-- Should not have triggers. > +#box.space._bucket:on_replace() > + | --- > + | - 0 > + | ... > + > +test_run:switch('router') > + | --- > + | - true > + | ... > +vshard.router.bootstrap() > + | --- > + | - true > + | ... > +vshard.router.callro(1, 'echo', {100}) > + | --- > + | - 100 > + | ... > + > +test_run:switch("default") > + | --- > + | - true > + | ... > +test_run:cmd('stop server router') > + | --- > + | - true > + | ... > +test_run:cmd('delete server router') > + | --- > + | - true > + | ... > +test_run:drop_cluster(REPLICASET_1) > + | --- > + | ... > diff --git a/test/router/boot_replica_first.test.lua b/test/router/boot_replica_first.test.lua > new file mode 100644 > index 0000000..7b1b3fd > --- /dev/null > +++ b/test/router/boot_replica_first.test.lua > @@ -0,0 +1,42 @@ > +test_run = require('test_run').new() > +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } > +test_run:create_cluster(REPLICASET_1, 'router', {args = 'boot_before_cfg'}) > +util = require('util') > +util.wait_master(test_run, REPLICASET_1, 'box_1_a') > +_ = test_run:cmd("create server router with script='router/router_2.lua'") > +_ = test_run:cmd("start server router") > + > +-- > +-- gh-237: replica should be able to boot before master. Before the issue was > +-- fixed, replica always tried to install a trigger on _bucket space even when > +-- it was not created on a master yet - that lead to an exception in > +-- storage.cfg. Now it should not install the trigger at all, because anyway it > +-- is not needed on replica for anything. > +-- > + > +test_run:switch('box_1_b') > +vshard.storage.cfg(cfg, instance_uuid) > +-- _bucket is not created yet. Will fail. > +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) > + > +test_run:switch('default') > +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') > + > +test_run:switch('box_1_a') > +vshard.storage.cfg(cfg, instance_uuid) > + > +test_run:switch('box_1_b') > +test_run:wait_lsn('box_1_b', 'box_1_a') > +-- Fails, but gracefully. Because the bucket is not found here. > +vshard.storage.call(1, 'read', 'echo', {100}) > +-- Should not have triggers. > +#box.space._bucket:on_replace() > + > +test_run:switch('router') > +vshard.router.bootstrap() > +vshard.router.callro(1, 'echo', {100}) > + > +test_run:switch("default") > +test_run:cmd('stop server router') > +test_run:cmd('delete server router') > +test_run:drop_cluster(REPLICASET_1) > diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua > index c6a78fe..5464824 100644 > --- a/vshard/storage/init.lua > +++ b/vshard/storage/init.lua > @@ -94,6 +94,17 @@ if not M then > -- detect that _bucket was not changed between yields. > -- > bucket_generation = 0, > + -- > + -- Reference to the function used as on_replace trigger on > + -- _bucket space. It is used to replace the trigger with > + -- a new function when reload happens. It is kept > + -- explicitly because the old function is deleted on > + -- reload from the global namespace. On the other hand, it > + -- is still stored in _bucket:on_replace() somewhere, but > + -- it is not known where. The only 100% way to be able to > + -- replace the old function is to keep its reference. > + -- > + bucket_on_replace = nil, > > ------------------- Garbage collection ------------------- > -- Fiber to remove garbage buckets data. > @@ -2435,8 +2446,14 @@ local function storage_cfg(cfg, this_replica_uuid, is_reload) > local uri = luri.parse(this_replica.uri) > schema_upgrade(is_master, uri.login, uri.password) > > - local old_trigger = box.space._bucket:on_replace()[1] > - box.space._bucket:on_replace(bucket_generation_increment, old_trigger) > + if M.bucket_on_replace then > + box.space._bucket:on_replace(nil, M.bucket_on_replace) > + M.bucket_on_replace = nil > + end > + if is_master then > + box.space._bucket:on_replace(bucket_generation_increment) > + M.bucket_on_replace = bucket_generation_increment > + end > > lreplicaset.rebind_replicasets(new_replicasets, M.replicasets) > lreplicaset.outdate_replicasets(M.replicasets) > diff --git a/vshard/storage/reload_evolution.lua b/vshard/storage/reload_evolution.lua > index 1abc9e2..f38af74 100644 > --- a/vshard/storage/reload_evolution.lua > +++ b/vshard/storage/reload_evolution.lua > @@ -17,6 +17,14 @@ migrations[#migrations + 1] = function(M) > -- Code to update Lua objects. > end > > +migrations[#migrations + 1] = function(M) > + local bucket = box.space._bucket > + if bucket then > + assert(M.bucket_on_replace == nil) > + M.bucket_on_replace = bucket:on_replace()[1] > + end > +end > + > -- > -- Perform an update based on a version stored in `M` (internals). > -- @param M Old module internals which should be updated. From yaroslav.dynnikov at tarantool.org Thu Aug 6 11:29:33 2020 From: yaroslav.dynnikov at tarantool.org (Yaroslav Dynnikov) Date: Thu, 6 Aug 2020 11:29:33 +0300 Subject: [Tarantool-patches] [PATCH v2 vshard 1/2] storage: fix reload applying migration twice In-Reply-To: References: Message-ID: Hi! Nice patchset, LGTM. Best regards Yaroslav Dynnikov On Thu, 6 Aug 2020 at 01:15, Vladislav Shpilevoy wrote: > Reload evolution is a subsystem for storage hot code reload. When > code is reloaded, it applies migration scripts one by one. But > there was a bug - the last migration script was applied even when > code version didn't really change. > > The bug didn't affect anything, since so far the reload evolution > was not used for anything and had just one dummy migration script > doing nothing. > > Now a first migration script is going to be added, so the bug is > fixed before that. > > Needed for #237 > --- > test/reload_evolution/storage.result | 6 +++++- > test/reload_evolution/storage.test.lua | 6 +++++- > vshard/storage/reload_evolution.lua | 2 +- > 3 files changed, 11 insertions(+), 3 deletions(-) > > diff --git a/test/reload_evolution/storage.result > b/test/reload_evolution/storage.result > index ebf4fdc..dde0a1f 100644 > --- a/test/reload_evolution/storage.result > +++ b/test/reload_evolution/storage.result > @@ -86,7 +86,11 @@ package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > --- > ... > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') ~= nil > +-- Should be nil. Because there was a bug that reload always reapplied > the last > +-- migration callback. When it was fixed, the last callback wasn't called > twice. > +-- But since the callback was only one, now nothing is called, and > nothing is > +-- logged. > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') == nil > --- > - true > ... > diff --git a/test/reload_evolution/storage.test.lua > b/test/reload_evolution/storage.test.lua > index 56c1693..7858112 100644 > --- a/test/reload_evolution/storage.test.lua > +++ b/test/reload_evolution/storage.test.lua > @@ -35,7 +35,11 @@ box.space.test:insert({42, bucket_id_to_move}) > package.path = original_package_path > package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') ~= nil > +-- Should be nil. Because there was a bug that reload always reapplied > the last > +-- migration callback. When it was fixed, the last callback wasn't called > twice. > +-- But since the callback was only one, now nothing is called, and > nothing is > +-- logged. > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') == nil > vshard.storage.internal.reload_version > -- Make sure storage operates well. > vshard.storage.bucket_force_drop(2000) > diff --git a/vshard/storage/reload_evolution.lua > b/vshard/storage/reload_evolution.lua > index 5d09b11..1abc9e2 100644 > --- a/vshard/storage/reload_evolution.lua > +++ b/vshard/storage/reload_evolution.lua > @@ -33,7 +33,7 @@ local function upgrade(M) > log.error(err_msg) > error(err_msg) > end > - for i = start_version, #migrations do > + for i = start_version + 1, #migrations do > local ok, err = pcall(migrations[i], M) > if ok then > log.info('vshard.storage.reload_evolution: upgraded to %d > version', > -- > 2.21.1 (Apple Git-122.3) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yaroslav.dynnikov at tarantool.org Thu Aug 6 11:30:12 2020 From: yaroslav.dynnikov at tarantool.org (Yaroslav Dynnikov) Date: Thu, 6 Aug 2020 11:30:12 +0300 Subject: [Tarantool-patches] [PATCH v2 vshard 2/2] storage: allow replica to boot before master In-Reply-To: <0cd777f3990939b32d96bf964dea55e8073e65bb.1596665591.git.v.shpilevoy@tarantool.org> References: <0cd777f3990939b32d96bf964dea55e8073e65bb.1596665591.git.v.shpilevoy@tarantool.org> Message-ID: Thanks for the patch, LGTM. Best regards Yaroslav Dynnikov On Thu, 6 Aug 2020 at 01:15, Vladislav Shpilevoy wrote: > Before the patch a replica couldn't call vshard.storage.cfg() > before the master is bootstrapped. That could lead to an exception > in the middle of vshard.storage.cfg(). > > It sounds not so bad and even looks unreachable, because replica > bootstrap anyway is blocked in box.cfg() until a master node is > started, otherwise the replica instance is terminated. > But the box.cfg bootstrap != vshard bootstrap. > > It could be that both replica and master nodes are already > bootstrapped and working, but vshard.storage.cfg() wasn't called > yet. In that case vshard.storage.cfg() on the replica wouldn't > block in box.cfg(), and would happily fail a few lines later in > vshard.storage.cfg() at attempt to touch not existing > box.space._bucket. > > That was the case for cartridge. The framework configures > instances in its own way before vshard.storage.cfg(). Then it > calls vshard.storage.cfg() on them in arbitrary order, and > sometimes replicas were configured earlier than the master. > > The fail is fixed by simply skipping the _bucket's trigger > installation on the replica node. Because so far it is not needed > here for anything. The trigger's sole purpose is to increment > bucket generation used only for DML on _bucket on the master node. > > Now vshard.storage.cfg() on replica is able to finish before > master does the same. However the replica still won't be able to > handle vshard.storage.* requests until receives vshard schema > from master. > > Closes #237 > --- > test/lua_libs/storage_template.lua | 55 +++++++++++- > test/misc/reconfigure.result | 33 +++++++ > test/misc/reconfigure.test.lua | 9 ++ > test/reload_evolution/storage.result | 16 ++-- > test/reload_evolution/storage.test.lua | 14 +-- > test/router/boot_replica_first.result | 112 ++++++++++++++++++++++++ > test/router/boot_replica_first.test.lua | 42 +++++++++ > vshard/storage/init.lua | 21 ++++- > vshard/storage/reload_evolution.lua | 8 ++ > 9 files changed, 297 insertions(+), 13 deletions(-) > create mode 100644 test/router/boot_replica_first.result > create mode 100644 test/router/boot_replica_first.test.lua > > diff --git a/test/lua_libs/storage_template.lua > b/test/lua_libs/storage_template.lua > index 29a753d..84e4180 100644 > --- a/test/lua_libs/storage_template.lua > +++ b/test/lua_libs/storage_template.lua > @@ -1,5 +1,7 @@ > #!/usr/bin/env tarantool > > +local luri = require('uri') > + > NAME = require('fio').basename(arg[0], '.lua') > fiber = require('fiber') > test_run = require('test_run').new() > @@ -10,12 +12,63 @@ log = require('log') > if not cfg.shard_index then > cfg.shard_index = 'bucket_id' > end > +instance_uuid = util.name_to_uuid[NAME] > + > +-- > +-- Bootstrap the instance exactly like vshard does. But don't > +-- initialize any vshard-specific code. > +-- > +local function boot_like_vshard() > + assert(type(box.cfg) == 'function') > + for rs_uuid, rs in pairs(cfg.sharding) do > + for replica_uuid, replica in pairs(rs.replicas) do > + if replica_uuid == instance_uuid then > + local box_cfg = {replication = {}} > + box_cfg.instance_uuid = replica_uuid > + box_cfg.replicaset_uuid = rs_uuid > + box_cfg.listen = replica.uri > + box_cfg.read_only = not replica.master > + box_cfg.replication_connect_quorum = 0 > + box_cfg.replication_timeout = 0.1 > + for _, replica in pairs(rs.replicas) do > + table.insert(box_cfg.replication, replica.uri) > + end > + box.cfg(box_cfg) > + if not replica.master then > + return > + end > + local uri = luri.parse(replica.uri) > + box.schema.user.create(uri.login, { > + password = uri.password, if_not_exists = true, > + }) > + box.schema.user.grant(uri.login, 'super') > + return > + end > + end > + end > + assert(false) > +end > + > +local omit_cfg = false > +local i = 1 > +while arg[i] ~= nil do > + local key = arg[i] > + i = i + 1 > + if key == 'boot_before_cfg' then > + boot_like_vshard() > + omit_cfg = true > + end > +end > > vshard = require('vshard') > echo_count = 0 > cfg.replication_connect_timeout = 3 > cfg.replication_timeout = 0.1 > -vshard.storage.cfg(cfg, util.name_to_uuid[NAME]) > + > +if not omit_cfg then > + vshard.storage.cfg(cfg, instance_uuid) > +end > + > function bootstrap_storage(engine) > box.once("testapp:schema:1", function() > if rawget(_G, 'CHANGE_SPACE_IDS') then > diff --git a/test/misc/reconfigure.result b/test/misc/reconfigure.result > index be58eca..168be5d 100644 > --- a/test/misc/reconfigure.result > +++ b/test/misc/reconfigure.result > @@ -277,6 +277,14 @@ box.cfg.replication > --- > - - storage:storage at 127.0.0.1:3301 > ... > +box.cfg.read_only > +--- > +- false > +... > +#box.space._bucket:on_replace() > +--- > +- 1 > +... > _ = test_run:switch('storage_2_a') > --- > ... > @@ -303,6 +311,15 @@ box.cfg.replication > - - storage:storage at 127.0.0.1:3303 > - storage:storage at 127.0.0.1:3304 > ... > +box.cfg.read_only > +--- > +- true > +... > +-- Should be zero on the slave node. Even though earlier the node was a > master. > +#box.space._bucket:on_replace() > +--- > +- 0 > +... > _ = test_run:switch('storage_2_b') > --- > ... > @@ -329,6 +346,14 @@ box.cfg.replication > - - storage:storage at 127.0.0.1:3303 > - storage:storage at 127.0.0.1:3304 > ... > +box.cfg.read_only > +--- > +- false > +... > +#box.space._bucket:on_replace() > +--- > +- 1 > +... > _ = test_run:switch('storage_3_a') > --- > ... > @@ -354,6 +379,14 @@ box.cfg.replication > --- > - - storage:storage at 127.0.0.1:3306 > ... > +box.cfg.read_only > +--- > +- false > +... > +#box.space._bucket:on_replace() > +--- > +- 1 > +... > _ = test_run:switch('router_1') > --- > ... > diff --git a/test/misc/reconfigure.test.lua > b/test/misc/reconfigure.test.lua > index 1b894c8..e891010 100644 > --- a/test/misc/reconfigure.test.lua > +++ b/test/misc/reconfigure.test.lua > @@ -111,6 +111,8 @@ for k,v in pairs(info.replicasets) do > table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +#box.space._bucket:on_replace() > > _ = test_run:switch('storage_2_a') > info = vshard.storage.info() > @@ -119,6 +121,9 @@ for k,v in pairs(info.replicasets) do > table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +-- Should be zero on the slave node. Even though earlier the node was a > master. > +#box.space._bucket:on_replace() > > _ = test_run:switch('storage_2_b') > info = vshard.storage.info() > @@ -127,6 +132,8 @@ for k,v in pairs(info.replicasets) do > table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +#box.space._bucket:on_replace() > > _ = test_run:switch('storage_3_a') > info = vshard.storage.info() > @@ -135,6 +142,8 @@ for k,v in pairs(info.replicasets) do > table.insert(uris, v.master.uri) end > table.sort(uris) > uris > box.cfg.replication > +box.cfg.read_only > +#box.space._bucket:on_replace() > > _ = test_run:switch('router_1') > info = vshard.router.info() > diff --git a/test/reload_evolution/storage.result > b/test/reload_evolution/storage.result > index dde0a1f..4652c4f 100644 > --- a/test/reload_evolution/storage.result > +++ b/test/reload_evolution/storage.result > @@ -86,16 +86,22 @@ package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > --- > ... > --- Should be nil. Because there was a bug that reload always reapplied > the last > --- migration callback. When it was fixed, the last callback wasn't called > twice. > --- But since the callback was only one, now nothing is called, and > nothing is > --- logged. > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') == nil > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') ~= nil > --- > - true > ... > vshard.storage.internal.reload_version > --- > +- 2 > +... > +-- > +-- gh-237: should be only one trigger. During gh-237 the trigger > installation > +-- became conditional and therefore required to remember the current > trigger > +-- somewhere. When reloaded from the old version, the trigger needed to be > +-- fetched from _bucket:on_replace(). > +-- > +#box.space._bucket:on_replace() > +--- > - 1 > ... > -- Make sure storage operates well. > diff --git a/test/reload_evolution/storage.test.lua > b/test/reload_evolution/storage.test.lua > index 7858112..06f7117 100644 > --- a/test/reload_evolution/storage.test.lua > +++ b/test/reload_evolution/storage.test.lua > @@ -35,12 +35,16 @@ box.space.test:insert({42, bucket_id_to_move}) > package.path = original_package_path > package.loaded['vshard.storage'] = nil > vshard.storage = require("vshard.storage") > --- Should be nil. Because there was a bug that reload always reapplied > the last > --- migration callback. When it was fixed, the last callback wasn't called > twice. > --- But since the callback was only one, now nothing is called, and > nothing is > --- logged. > -test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') == nil > +test_run:grep_log('storage_2_a', 'vshard.storage.reload_evolution: > upgraded to') ~= nil > vshard.storage.internal.reload_version > +-- > +-- gh-237: should be only one trigger. During gh-237 the trigger > installation > +-- became conditional and therefore required to remember the current > trigger > +-- somewhere. When reloaded from the old version, the trigger needed to be > +-- fetched from _bucket:on_replace(). > +-- > +#box.space._bucket:on_replace() > + > -- Make sure storage operates well. > vshard.storage.bucket_force_drop(2000) > vshard.storage.bucket_force_create(2000) > diff --git a/test/router/boot_replica_first.result > b/test/router/boot_replica_first.result > new file mode 100644 > index 0000000..1705230 > --- /dev/null > +++ b/test/router/boot_replica_first.result > @@ -0,0 +1,112 @@ > +-- test-run result file version 2 > +test_run = require('test_run').new() > + | --- > + | ... > +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } > + | --- > + | ... > +test_run:create_cluster(REPLICASET_1, 'router', {args = > 'boot_before_cfg'}) > + | --- > + | ... > +util = require('util') > + | --- > + | ... > +util.wait_master(test_run, REPLICASET_1, 'box_1_a') > + | --- > + | ... > +_ = test_run:cmd("create server router with script='router/router_2.lua'") > + | --- > + | ... > +_ = test_run:cmd("start server router") > + | --- > + | ... > + > +-- > +-- gh-237: replica should be able to boot before master. Before the issue > was > +-- fixed, replica always tried to install a trigger on _bucket space even > when > +-- it was not created on a master yet - that lead to an exception in > +-- storage.cfg. Now it should not install the trigger at all, because > anyway it > +-- is not needed on replica for anything. > +-- > + > +test_run:switch('box_1_b') > + | --- > + | - true > + | ... > +vshard.storage.cfg(cfg, instance_uuid) > + | --- > + | ... > +-- _bucket is not created yet. Will fail. > +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) > + | --- > + | - attempt to index field '_bucket' (a nil value) > + | ... > + > +test_run:switch('default') > + | --- > + | - true > + | ... > +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') > + | --- > + | ... > + > +test_run:switch('box_1_a') > + | --- > + | - true > + | ... > +vshard.storage.cfg(cfg, instance_uuid) > + | --- > + | ... > + > +test_run:switch('box_1_b') > + | --- > + | - true > + | ... > +test_run:wait_lsn('box_1_b', 'box_1_a') > + | --- > + | ... > +-- Fails, but gracefully. Because the bucket is not found here. > +vshard.storage.call(1, 'read', 'echo', {100}) > + | --- > + | - null > + | - bucket_id: 1 > + | reason: Not found > + | code: 1 > + | type: ShardingError > + | message: 'Cannot perform action with bucket 1, reason: Not found' > + | name: WRONG_BUCKET > + | ... > +-- Should not have triggers. > +#box.space._bucket:on_replace() > + | --- > + | - 0 > + | ... > + > +test_run:switch('router') > + | --- > + | - true > + | ... > +vshard.router.bootstrap() > + | --- > + | - true > + | ... > +vshard.router.callro(1, 'echo', {100}) > + | --- > + | - 100 > + | ... > + > +test_run:switch("default") > + | --- > + | - true > + | ... > +test_run:cmd('stop server router') > + | --- > + | - true > + | ... > +test_run:cmd('delete server router') > + | --- > + | - true > + | ... > +test_run:drop_cluster(REPLICASET_1) > + | --- > + | ... > diff --git a/test/router/boot_replica_first.test.lua > b/test/router/boot_replica_first.test.lua > new file mode 100644 > index 0000000..7b1b3fd > --- /dev/null > +++ b/test/router/boot_replica_first.test.lua > @@ -0,0 +1,42 @@ > +test_run = require('test_run').new() > +REPLICASET_1 = { 'box_1_a', 'box_1_b', 'box_1_c' } > +test_run:create_cluster(REPLICASET_1, 'router', {args = > 'boot_before_cfg'}) > +util = require('util') > +util.wait_master(test_run, REPLICASET_1, 'box_1_a') > +_ = test_run:cmd("create server router with script='router/router_2.lua'") > +_ = test_run:cmd("start server router") > + > +-- > +-- gh-237: replica should be able to boot before master. Before the issue > was > +-- fixed, replica always tried to install a trigger on _bucket space even > when > +-- it was not created on a master yet - that lead to an exception in > +-- storage.cfg. Now it should not install the trigger at all, because > anyway it > +-- is not needed on replica for anything. > +-- > + > +test_run:switch('box_1_b') > +vshard.storage.cfg(cfg, instance_uuid) > +-- _bucket is not created yet. Will fail. > +util.check_error(vshard.storage.call, 1, 'read', 'echo', {100}) > + > +test_run:switch('default') > +util.map_evals(test_run, {REPLICASET_1}, 'bootstrap_storage(\'memtx\')') > + > +test_run:switch('box_1_a') > +vshard.storage.cfg(cfg, instance_uuid) > + > +test_run:switch('box_1_b') > +test_run:wait_lsn('box_1_b', 'box_1_a') > +-- Fails, but gracefully. Because the bucket is not found here. > +vshard.storage.call(1, 'read', 'echo', {100}) > +-- Should not have triggers. > +#box.space._bucket:on_replace() > + > +test_run:switch('router') > +vshard.router.bootstrap() > +vshard.router.callro(1, 'echo', {100}) > + > +test_run:switch("default") > +test_run:cmd('stop server router') > +test_run:cmd('delete server router') > +test_run:drop_cluster(REPLICASET_1) > diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua > index c6a78fe..5464824 100644 > --- a/vshard/storage/init.lua > +++ b/vshard/storage/init.lua > @@ -94,6 +94,17 @@ if not M then > -- detect that _bucket was not changed between yields. > -- > bucket_generation = 0, > + -- > + -- Reference to the function used as on_replace trigger on > + -- _bucket space. It is used to replace the trigger with > + -- a new function when reload happens. It is kept > + -- explicitly because the old function is deleted on > + -- reload from the global namespace. On the other hand, it > + -- is still stored in _bucket:on_replace() somewhere, but > + -- it is not known where. The only 100% way to be able to > + -- replace the old function is to keep its reference. > + -- > + bucket_on_replace = nil, > > ------------------- Garbage collection ------------------- > -- Fiber to remove garbage buckets data. > @@ -2435,8 +2446,14 @@ local function storage_cfg(cfg, this_replica_uuid, > is_reload) > local uri = luri.parse(this_replica.uri) > schema_upgrade(is_master, uri.login, uri.password) > > - local old_trigger = box.space._bucket:on_replace()[1] > - box.space._bucket:on_replace(bucket_generation_increment, old_trigger) > + if M.bucket_on_replace then > + box.space._bucket:on_replace(nil, M.bucket_on_replace) > + M.bucket_on_replace = nil > + end > + if is_master then > + box.space._bucket:on_replace(bucket_generation_increment) > + M.bucket_on_replace = bucket_generation_increment > + end > > lreplicaset.rebind_replicasets(new_replicasets, M.replicasets) > lreplicaset.outdate_replicasets(M.replicasets) > diff --git a/vshard/storage/reload_evolution.lua > b/vshard/storage/reload_evolution.lua > index 1abc9e2..f38af74 100644 > --- a/vshard/storage/reload_evolution.lua > +++ b/vshard/storage/reload_evolution.lua > @@ -17,6 +17,14 @@ migrations[#migrations + 1] = function(M) > -- Code to update Lua objects. > end > > +migrations[#migrations + 1] = function(M) > + local bucket = box.space._bucket > + if bucket then > + assert(M.bucket_on_replace == nil) > + M.bucket_on_replace = bucket:on_replace()[1] > + end > +end > + > -- > -- Perform an update based on a version stored in `M` (internals). > -- @param M Old module internals which should be updated. > -- > 2.21.1 (Apple Git-122.3) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yaroslav.dynnikov at tarantool.org Thu Aug 6 16:19:55 2020 From: yaroslav.dynnikov at tarantool.org (Yaroslav Dynnikov) Date: Thu, 6 Aug 2020 16:19:55 +0300 Subject: [Tarantool-patches] [PATCH] Ensure all curl symbols are exported Message-ID: <20200806131955.3400088-1-yaroslav.dynnikov@tarantool.org> In the recent update of libcurl (2.5.0-278-g807c7fa58) its layout has changed: private function `Curl_version_init()` which used to fill-in info structure was eliminated. As a result, no symbols for `libcurl_la-version.o` remained used, so it wasn't included in tarantool binary. And `curl_version` and `curl_version_info` symbols went missing. According to libcurl naming conventions all exported symbols are named as `curl_*`. This patch lists them all explicitly in `exprots.h` and adds the test. Close #5223 Issue: https://github.com/tarantool/tarantool/issues/5223 Branch: https://github.com/tarantool/tarantool/tree/rosik/gh-5223-missing-curl-symbols --- src/exports.h | 81 ++++++++++ test/box-tap/gh-5223-curl-exports.test.lua | 177 +++++++++++++++++++++ 2 files changed, 258 insertions(+) create mode 100755 test/box-tap/gh-5223-curl-exports.test.lua diff --git a/src/exports.h b/src/exports.h index 7cf283e5b..7f0601f4f 100644 --- a/src/exports.h +++ b/src/exports.h @@ -113,6 +113,87 @@ EXPORT(csv_feed) EXPORT(csv_iterator_create) EXPORT(csv_next) EXPORT(csv_setopt) +EXPORT(curl_easy_cleanup) +EXPORT(curl_easy_duphandle) +EXPORT(curl_easy_escape) +EXPORT(curl_easy_getinfo) +EXPORT(curl_easy_init) +EXPORT(curl_easy_pause) +EXPORT(curl_easy_perform) +EXPORT(curl_easy_recv) +EXPORT(curl_easy_reset) +EXPORT(curl_easy_send) +EXPORT(curl_easy_setopt) +EXPORT(curl_easy_strerror) +EXPORT(curl_easy_unescape) +EXPORT(curl_easy_upkeep) +EXPORT(curl_escape) +EXPORT(curl_formadd) +EXPORT(curl_formfree) +EXPORT(curl_formget) +EXPORT(curl_free) +EXPORT(curl_getdate) +EXPORT(curl_getenv) +EXPORT(curl_global_cleanup) +EXPORT(curl_global_init) +EXPORT(curl_global_init_mem) +EXPORT(curl_global_sslset) +EXPORT(curl_maprintf) +EXPORT(curl_mfprintf) +EXPORT(curl_mime_addpart) +EXPORT(curl_mime_data) +EXPORT(curl_mime_data_cb) +EXPORT(curl_mime_encoder) +EXPORT(curl_mime_filedata) +EXPORT(curl_mime_filename) +EXPORT(curl_mime_free) +EXPORT(curl_mime_headers) +EXPORT(curl_mime_init) +EXPORT(curl_mime_name) +EXPORT(curl_mime_subparts) +EXPORT(curl_mime_type) +EXPORT(curl_mprintf) +EXPORT(curl_msnprintf) +EXPORT(curl_msprintf) +EXPORT(curl_multi_add_handle) +EXPORT(curl_multi_assign) +EXPORT(curl_multi_cleanup) +EXPORT(curl_multi_fdset) +EXPORT(curl_multi_info_read) +EXPORT(curl_multi_init) +EXPORT(curl_multi_perform) +EXPORT(curl_multi_poll) +EXPORT(curl_multi_remove_handle) +EXPORT(curl_multi_setopt) +EXPORT(curl_multi_socket) +EXPORT(curl_multi_socket_action) +EXPORT(curl_multi_socket_all) +EXPORT(curl_multi_strerror) +EXPORT(curl_multi_timeout) +EXPORT(curl_multi_wait) +EXPORT(curl_mvaprintf) +EXPORT(curl_mvfprintf) +EXPORT(curl_mvprintf) +EXPORT(curl_mvsnprintf) +EXPORT(curl_mvsprintf) +EXPORT(curl_pushheader_byname) +EXPORT(curl_pushheader_bynum) +EXPORT(curl_share_cleanup) +EXPORT(curl_share_init) +EXPORT(curl_share_setopt) +EXPORT(curl_share_strerror) +EXPORT(curl_slist_append) +EXPORT(curl_slist_free_all) +EXPORT(curl_strequal) +EXPORT(curl_strnequal) +EXPORT(curl_unescape) +EXPORT(curl_url) +EXPORT(curl_url_cleanup) +EXPORT(curl_url_dup) +EXPORT(curl_url_get) +EXPORT(curl_url_set) +EXPORT(curl_version) +EXPORT(curl_version_info) EXPORT(decimal_unpack) EXPORT(error_ref) EXPORT(error_set_prev) diff --git a/test/box-tap/gh-5223-curl-exports.test.lua b/test/box-tap/gh-5223-curl-exports.test.lua new file mode 100755 index 000000000..300d60b07 --- /dev/null +++ b/test/box-tap/gh-5223-curl-exports.test.lua @@ -0,0 +1,177 @@ +#!/usr/bin/env tarantool + +local tap = require('tap') +local ffi = require('ffi') +ffi.cdef([[ + void *dlsym(void *handle, const char *symbol); + struct curl_version_info_data { + int age; /* see description below */ + const char *version; /* human readable string */ + unsigned int version_num; /* numeric representation */ + const char *host; /* human readable string */ + int features; /* bitmask, see below */ + char *ssl_version; /* human readable string */ + long ssl_version_num; /* not used, always zero */ + const char *libz_version; /* human readable string */ + const char * const *protocols; /* protocols */ + + /* when 'age' is CURLVERSION_SECOND or higher, the members below exist */ + const char *ares; /* human readable string */ + int ares_num; /* number */ + + /* when 'age' is CURLVERSION_THIRD or higher, the members below exist */ + const char *libidn; /* human readable string */ + + /* when 'age' is CURLVERSION_FOURTH or higher (>= 7.16.1), the members + below exist */ + int iconv_ver_num; /* '_libiconv_version' if iconv support enabled */ + + const char *libssh_version; /* human readable string */ + + /* when 'age' is CURLVERSION_FIFTH or higher (>= 7.57.0), the members + below exist */ + unsigned int brotli_ver_num; /* Numeric Brotli version + (MAJOR << 24) | (MINOR << 12) | PATCH */ + const char *brotli_version; /* human readable string. */ + + /* when 'age' is CURLVERSION_SIXTH or higher (>= 7.66.0), the members + below exist */ + unsigned int nghttp2_ver_num; /* Numeric nghttp2 version + (MAJOR << 16) | (MINOR << 8) | PATCH */ + const char *nghttp2_version; /* human readable string. */ + + const char *quic_version; /* human readable quic (+ HTTP/3) library + + version or NULL */ + + /* when 'age' is CURLVERSION_SEVENTH or higher (>= 7.70.0), the members + below exist */ + const char *cainfo; /* the built-in default CURLOPT_CAINFO, might + be NULL */ + const char *capath; /* the built-in default CURLOPT_CAPATH, might + be NULL */ + }; + + struct curl_version_info_data *curl_version_info(int age); +]]) + +local info = ffi.C.curl_version_info(7) +local test = tap.test('curl-features') +test:plan(3) + +if test:ok(info.ssl_version ~= nil, 'Curl built with SSL support') then + test:diag('ssl_version: ' .. ffi.string(info.ssl_version)) +end +if test:ok(info.libz_version ~= nil, 'Curl built with LIBZ') then + test:diag('libz_version: ' .. ffi.string(info.libz_version)) +end + +local RTLD_DEFAULT +-- See `man 3 dlsym`: +-- RTLD_DEFAULT +-- Find the first occurrence of the desired symbol using the default +-- shared object search order. The search will include global symbols +-- in the executable and its dependencies, as well as symbols in shared +-- objects that were dynamically loaded with the RTLD_GLOBAL flag. +if jit.os == "OSX" then + RTLD_DEFAULT = ffi.cast("void *", -2LL) +else + RTLD_DEFAULT = ffi.cast("void *", 0LL) +end + +-- The following list was obtained by parsing libcurl.a static library: +-- nm libcurl.a | grep -oP 'T \K(curl_.+)$' | sort +local curl_symbols = { + 'curl_easy_cleanup', + 'curl_easy_duphandle', + 'curl_easy_escape', + 'curl_easy_getinfo', + 'curl_easy_init', + 'curl_easy_pause', + 'curl_easy_perform', + 'curl_easy_recv', + 'curl_easy_reset', + 'curl_easy_send', + 'curl_easy_setopt', + 'curl_easy_strerror', + 'curl_easy_unescape', + 'curl_easy_upkeep', + 'curl_escape', + 'curl_formadd', + 'curl_formfree', + 'curl_formget', + 'curl_free', + 'curl_getdate', + 'curl_getenv', + 'curl_global_cleanup', + 'curl_global_init', + 'curl_global_init_mem', + 'curl_global_sslset', + 'curl_maprintf', + 'curl_mfprintf', + 'curl_mime_addpart', + 'curl_mime_data', + 'curl_mime_data_cb', + 'curl_mime_encoder', + 'curl_mime_filedata', + 'curl_mime_filename', + 'curl_mime_free', + 'curl_mime_headers', + 'curl_mime_init', + 'curl_mime_name', + 'curl_mime_subparts', + 'curl_mime_type', + 'curl_mprintf', + 'curl_msnprintf', + 'curl_msprintf', + 'curl_multi_add_handle', + 'curl_multi_assign', + 'curl_multi_cleanup', + 'curl_multi_fdset', + 'curl_multi_info_read', + 'curl_multi_init', + 'curl_multi_perform', + 'curl_multi_poll', + 'curl_multi_remove_handle', + 'curl_multi_setopt', + 'curl_multi_socket', + 'curl_multi_socket_action', + 'curl_multi_socket_all', + 'curl_multi_strerror', + 'curl_multi_timeout', + 'curl_multi_wait', + 'curl_mvaprintf', + 'curl_mvfprintf', + 'curl_mvprintf', + 'curl_mvsnprintf', + 'curl_mvsprintf', + 'curl_pushheader_byname', + 'curl_pushheader_bynum', + 'curl_share_cleanup', + 'curl_share_init', + 'curl_share_setopt', + 'curl_share_strerror', + 'curl_slist_append', + 'curl_slist_free_all', + 'curl_strequal', + 'curl_strnequal', + 'curl_unescape', + 'curl_url', + 'curl_url_cleanup', + 'curl_url_dup', + 'curl_url_get', + 'curl_url_set', + 'curl_version', + 'curl_version_info', +} + +test:test('curl_symbols', function(t) + t:plan(#curl_symbols) + for _, sym in ipairs(curl_symbols) do + t:ok( + ffi.C.dlsym(RTLD_DEFAULT, sym) ~= nil, + ('Symbol %q found'):format(sym) + ) + end +end) + +os.exit(test:check() and 0 or 1) -- 2.25.1 From huston.mavr at gmail.com Thu Aug 6 16:32:32 2020 From: huston.mavr at gmail.com (Alexandr Barulev) Date: Thu, 6 Aug 2020 16:32:32 +0300 Subject: [Tarantool-patches] [PATCH] build: refactor static build process In-Reply-To: References: <20200622181649.10100-1-huston.mavr@gmail.com> <20200727223734.cnxrdzik2cyt3ey4@tkn_work_nb> Message-ID: I?ve squashed commit and changed it?s message; Also I?ve sended diff at previous answer https://github.com/tarantool/tarantool/tree/rosik/refactor-static-build ??, 5 ???. 2020 ?. ? 20:08, Mavr Huston : > Hi, thanks for the review! > > libicu installs as ExternalProject_Add too, its missed in commit message; > > Problem with curses and ncurses was on macOS and linux, because libcurses > is an entire copy of libncurses, and tarantool links with system > libcurses instead of libncurses installed as tarantool dependency, but > module FindCurses.cmkae provides workaround for this problem - > CURSES_NEED_NCURSES flag.- to use ncurses instead of curses. (i will fix > this part at commit message) > > About disable-shred flag, used at libcurl building - we want to link only > with > static libraries, so we prevent creating unused .so. > > I've renamed static_build_no_deps_* jobs after review to > static_build_cmake_* > > Also about such path tarantool-prefix/* - it's a cmake > ExternalProject_Add() > default path (i've also added comment at .travis.mk) > > Useless comments "Init macOS test env" deleted. > > > if (BUILD_STATIC) > > - set(LIBZ_LIB_NAME libz.a) > > + find_library(LIBZ_LIBRARY NAMES libz.a) > > else() > > - set(LIBZ_LIB_NAME z) > > + find_library(LIBZ_LIBRARY NAMES z) > > endif() > > - find_library(LIBZ_LIBRARY NAMES ${LIBZ_LIB_NAME}) > Here we simplified code, by deleting useless variable. > > I've added commentaries to cmake/compiler.cmake about libunwind on macOS > and about ignoring flag -static-libstdc++ on macOS > > I've fixed static-build for using system compiler: gcc/g++ on linux > and clang/clang++ on macOS > > I've refactored IF (NOT APPLE) condition to IF (APPLE) at > static-build/CMakeLists.txt > > I've mentioned macOS dependencies at static-build/README.md xcode-tools and > others, also I've added example with CMAKE_TARANTOOL_ARGS. > > Added commentaries about _EP_INSTALL_DIR at static-build/CMakeLists.txt > > Also deleted unused use_unix_sockets_iproto = True > > Also deleted curl-features.test.lua, because after rebase on master it > fails, > due to missing curl_version_info symbol at tarantool binary. This symbol > lost > after #807c7fa584f21ee955b2a14623d70f7510a3650d (build: update curl > submodule > to 7.71.1 version ) > > > After pass the review I'll squash this changes to base commit and update > commit > message. > > Here is a diff of changes: > ===================================== > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 5ec5dd9b9..c9aef3dc7 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -534,14 +534,14 @@ static_build: > script: > - ${GITLAB_MAKE} test_static_build > > -static_build_no_deps_linux: > +static_build_cmake_linux: > <<: *docker_test_definition > script: > - - ${GITLAB_MAKE} test_static_build_no_deps_linux > + - ${GITLAB_MAKE} test_static_build_cmake_linux > > -static_build_no_deps_osx_15: > +static_build_cmake_osx_15: > stage: test > tags: > - osx_15 > script: > - - ${GITLAB_MAKE} test_static_build_no_deps_osx > + - ${GITLAB_MAKE} test_static_build_cmake_osx > diff --git a/.travis.mk b/.travis.mk > index 64862348f..482672429 100644 > --- a/.travis.mk > +++ b/.travis.mk > @@ -149,8 +149,8 @@ test_static_build: deps_debian_static > CMAKE_EXTRA_PARAMS=-DBUILD_STATIC=ON make -f .travis.mk > test_debian_no_deps > > # New static build > - > -test_static_build_no_deps_linux: > +# builddir used in this target - is a default build path from cmake > ExternalProject_Add() > +test_static_build_cmake_linux: > cd static-build && cmake . && make -j && ctest -V > cd test && /usr/bin/python test-run.py --force \ > --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build > $(TEST_RUN_EXTRA_PARAMS) > @@ -218,7 +218,6 @@ INIT_TEST_ENV_OSX=\ > rm -rf /tmp/tnt > > test_osx_no_deps: build_osx > - # Init macOS test env > ${INIT_TEST_ENV_OSX}; \ > cd test && ./test-run.py --vardir /tmp/tnt --force > $(TEST_RUN_EXTRA_PARAMS) > > @@ -233,9 +232,9 @@ base_deps_osx: > brew install --force ${STATIC_OSX_PKGS} || brew upgrade > ${STATIC_OSX_PKGS} > pip install --force-reinstall -r test-run/requirements.txt > > -test_static_build_no_deps_osx: base_deps_osx > +# builddir used in this target - is a default build path from cmake > ExternalProject_Add() > +test_static_build_cmake_osx: base_deps_osx > cd static-build && cmake . && make -j && ctest -V > - # Init macOS test env > ${INIT_TEST_ENV_OSX}; \ > cd test && ./test-run.py --vardir /tmp/tnt \ > --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build \ > diff --git a/cmake/FindReadline.cmake b/cmake/FindReadline.cmake > index 8422181d6..afe480679 100644 > --- a/cmake/FindReadline.cmake > +++ b/cmake/FindReadline.cmake > @@ -14,6 +14,14 @@ if(BUILD_STATIC) > if (NOT CURSES_INFO_LIBRARY) > set(CURSES_INFO_LIBRARY "") > endif() > + > + # From Modules/FindCurses.cmake: > + # Set ``CURSES_NEED_NCURSES`` to ``TRUE`` before the > + # ``find_package(Curses)`` call if NCurses functionality is required. > + # This flag is set for linking with required library (installed > + # via static-build/CMakeLists.txt). If this variable won't be set > + # then tarantool binary links with system library curses which is an > + # entire copy of ncurses > set(CURSES_NEED_NCURSES TRUE) > endif() > > diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake > index 14f1e1186..db2ae6227 100644 > --- a/cmake/compiler.cmake > +++ b/cmake/compiler.cmake > @@ -131,6 +131,8 @@ set(CMAKE_REQUIRED_INCLUDES "") > if(BUILD_STATIC AND NOT TARGET_OS_DARWIN) > set(UNWIND_LIB_NAME libunwind.a) > else() > + # libunwind can't be compiled on macOS. > + # But there exists libunwind.dylib as a part of MacOSSDK > set(UNWIND_LIB_NAME unwind) > endif() > find_library(UNWIND_LIBRARY PATH_SUFFIXES system NAMES ${UNWIND_LIB_NAME}) > @@ -192,6 +194,9 @@ if (ENABLE_BACKTRACE) > find_package_message(UNWIND_LIBRARIES "Found unwind" > "${UNWIND_LIBRARIES}") > endif() > > +# On macOS there is no '-static-libstdc++' flag and it's use will > +# raise following error: > +# error: argument unused during compilation: '-static-libstdc++' > if(BUILD_STATIC AND NOT TARGET_OS_DARWIN) > # Static linking for c++ routines > add_compile_flags("C;CXX" "-static-libstdc++") > diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt > index 86582af0a..53ceb609c 100644 > --- a/static-build/CMakeLists.txt > +++ b/static-build/CMakeLists.txt > @@ -9,11 +9,18 @@ set(NCURSES_VERSION 6.2) > set(READLINE_VERSION 8.0) > set(UNWIND_VERSION 1.3-rc1) > > -find_program(C_COMPILER gcc) > -find_program(CXX_COMPILER g++) > +if (APPLE) > + find_program(C_COMPILER clang) > + find_program(CXX_COMPILER clang++) > +else() > + find_program(C_COMPILER gcc) > + find_program(CXX_COMPILER g++) > +endif() > set(CMAKE_C_COMPILER ${C_COMPILER}) > set(CMAKE_CXX_COMPILER ${CXX_COMPILER}) > > +# Install all libraries required by tarantool at current build dir > + > # > # OpenSSL > # > @@ -80,7 +87,18 @@ ExternalProject_Add(readline > # > # ICONV > # > -if (NOT APPLE) > +if (APPLE) > + ExternalProject_Add(iconv > + URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz > + CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER} > + /configure > + --prefix= > + --disable-shared > + --enable-static > + --with-gnu-ld > + STEP_TARGETS download > + ) > +else() > # In linux iconv is embedded into glibc > # So we find system header and copy it locally > find_path(ICONV_INCLUDE_DIR iconv.h) > @@ -101,20 +119,11 @@ if (NOT APPLE) > add_custom_target(iconv > DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix/include/iconv.h" > ) > + # This is a hack for further getting install directory of library > + # by ExternalProject_Get_Property > set_target_properties(iconv > PROPERTIES _EP_INSTALL_DIR ${ICONV_INSTALL_PREFIX} > ) > -else() > - ExternalProject_Add(iconv > - URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz > - CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER} > - /configure > - --prefix= > - --disable-shared > - --enable-static > - --with-gnu-ld > - STEP_TARGETS download > - ) > endif() > > # > @@ -162,6 +171,8 @@ if (APPLE) > endif() > > add_custom_target(unwind DEPENDS ${UNWIND_DEPENDENCIES}) > + # This is a hack for further getting install directory of library > + # by ExternalProject_Get_Property > set_target_properties(unwind > PROPERTIES _EP_INSTALL_DIR ${UNWIND_INSTALL_PREFIX} > ) > @@ -178,6 +189,8 @@ else() > ) > endif() > > +# Get install directories of builded libraries for building > +# tarantool with custon CMAKE_PREFIX_PATH > foreach(PROJ openssl icu zlib ncurses readline iconv unwind) > ExternalProject_Get_Property(${PROJ} install_dir) > set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH}:${install_dir}) > @@ -197,16 +210,14 @@ ExternalProject_Add(tarantool > -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH} > -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=FALSE > -DOPENSSL_USE_STATIC_LIBS=TRUE > - -DCMAKE_BUILD_TYPE=Debug > -DBUILD_STATIC=TRUE > -DENABLE_DIST=TRUE > -DENABLE_BACKTRACE=TRUE > - -DPACKAGE:STRING=${PACKAGE_NAME} > -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} > -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} > ${CMAKE_TARANTOOL_ARGS} > - BUILD_COMMAND ${CMAKE_MAKE_PROGRAM} -j > STEP_TARGETS build > + BUILD_COMMAND $(MAKE) > ) > > enable_testing() > diff --git a/static-build/README.md b/static-build/README.md > index 29fe085c3..0019e963f 100644 > --- a/static-build/README.md > +++ b/static-build/README.md > @@ -13,6 +13,24 @@ yum install -y \ > python-msgpack python-yaml python-argparse python-six python-gevent > ``` > > +MacOS: > + > +Before you start please install default Xcode Tools by Apple: > + > +``` > +sudo xcode-select --install > +sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer > +``` > + > +Install brew using command from > +[Homebrew repository instructions](https://github.com/Homebrew/inst) > + > +After that run next script: > + > +```bash > + brew install autoconf automake libtool cmake > file://$${PWD}/tools/brew_taps/tntpython2.rbs > + pip install --force-reinstall -r test-run/requirements.txt > +``` > > ### Usage > > @@ -21,3 +39,20 @@ cmake . > make -j > ctest -V > ``` > + > +## Customize your build > + > +If you want to customise build, you need to set `CMAKE_TARANTOOL_ARGS` > variable > + > +### Usage > + > +There is three types of `CMAKE_BUILD_TYPE`: > +* Debug - default > +* Release > +* RelWithDebInfo > + > +And you want to build tarantool with RelWithDebInfo: > + > +```bash > +cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo" . > +``` > diff --git a/static-build/test/static-build/curl-features.test.lua > b/static-build/test/static-build/curl-features.test.lua > deleted file mode 100755 > index 57b1c4306..000000000 > --- a/static-build/test/static-build/curl-features.test.lua > +++ /dev/null > @@ -1,67 +0,0 @@ > -#!/usr/bin/env tarantool > - > -local tap = require('tap') > -local ffi = require('ffi') > -ffi.cdef([[ > - struct curl_version_info_data { > - int age; /* see description below */ > - const char *version; /* human readable string */ > - unsigned int version_num; /* numeric representation */ > - const char *host; /* human readable string */ > - int features; /* bitmask, see below */ > - char *ssl_version; /* human readable string */ > - long ssl_version_num; /* not used, always zero */ > - const char *libz_version; /* human readable string */ > - const char * const *protocols; /* protocols */ > - > - /* when 'age' is CURLVERSION_SECOND or higher, the members below > exist */ > - const char *ares; /* human readable string */ > - int ares_num; /* number */ > - > - /* when 'age' is CURLVERSION_THIRD or higher, the members below > exist */ > - const char *libidn; /* human readable string */ > - > - /* when 'age' is CURLVERSION_FOURTH or higher (>= 7.16.1), the > members > - below exist */ > - int iconv_ver_num; /* '_libiconv_version' if iconv support > enabled */ > - > - const char *libssh_version; /* human readable string */ > - > - /* when 'age' is CURLVERSION_FIFTH or higher (>= 7.57.0), the > members > - below exist */ > - unsigned int brotli_ver_num; /* Numeric Brotli version > - (MAJOR << 24) | (MINOR << 12) | > PATCH */ > - const char *brotli_version; /* human readable string. */ > - > - /* when 'age' is CURLVERSION_SIXTH or higher (>= 7.66.0), the > members > - below exist */ > - unsigned int nghttp2_ver_num; /* Numeric nghttp2 version > - (MAJOR << 16) | (MINOR << 8) | > PATCH */ > - const char *nghttp2_version; /* human readable string. */ > - > - const char *quic_version; /* human readable quic (+ HTTP/3) > library + > - version or NULL */ > - > - /* when 'age' is CURLVERSION_SEVENTH or higher (>= 7.70.0), the > members > - below exist */ > - const char *cainfo; /* the built-in default > CURLOPT_CAINFO, might > - be NULL */ > - const char *capath; /* the built-in default > CURLOPT_CAPATH, might > - be NULL */ > - }; > - > - struct curl_version_info_data *curl_version_info(int age); > -]]) > - > -local info = ffi.C.curl_version_info(7) > -local test = tap.test('curl-features') > -test:plan(2) > - > -if test:ok(info.ssl_version ~= nil, 'Curl built with SSL support') then > - test:diag('ssl_version: ' .. ffi.string(info.ssl_version)) > -end > -if test:ok(info.libz_version ~= nil, 'Curl built with LIBZ') then > - test:diag('libz_version: ' .. ffi.string(info.libz_version)) > -end > - > -os.exit(test:check() and 0 or 1) > diff --git a/static-build/test/static-build/suite.ini > b/static-build/test/static-build/suite.ini > index 4da3d5d2f..92e349466 100644 > --- a/static-build/test/static-build/suite.ini > +++ b/static-build/test/static-build/suite.ini > @@ -3,4 +3,3 @@ core = app > description = Static build tests > script = box.lua > is_parallel = True > -use_unix_sockets_iproto = True > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imun at tarantool.org Thu Aug 6 17:41:53 2020 From: imun at tarantool.org (Igor Munkin) Date: Thu, 6 Aug 2020 17:41:53 +0300 Subject: [Tarantool-patches] [V2] luajit: pass properly compile options to LuaJIT for RelWithDebInfo In-Reply-To: References: Message-ID: <20200806144153.GW18920@tarantool.org> Timur, Thanks for the patch! Please, consider my comments below. On 24.07.20, Timur Safin wrote: > It was discovered the harder way (while debugging) that LuaJIT > sources do not generate line info debug information while we are Typo (still on the remote branch): s/whil/while/. Typo (still on the remote branch): s/we in/we are in/. > in any of optimized modes (e.g. RelWithDebInfo). > > Simplistically we need to append `-g -ggdb` to optimization > flags in luajit_cflags, but that would be fragile in the longer > run. We need just to pass CMAKE_C_FLAGS_ instead, Well, you claim that the simple solution is fragile, but I don't see the reason. Could you please clarify your concerns? > because it would contain -g -ggdb for RelWithDebInfo, iff Typo: s/iff/if/. > selected compiler support those options. Typo: s/support/supports/. > > Closes #4827 > --- > cmake/luajit.cmake | 17 ++++++----------- > 1 file changed, 6 insertions(+), 11 deletions(-) > > diff --git a/cmake/luajit.cmake b/cmake/luajit.cmake > index 555bc8371..76155c0c6 100644 > --- a/cmake/luajit.cmake > +++ b/cmake/luajit.cmake > @@ -139,7 +139,9 @@ macro(luajit_build) > # will most certainly wreak havok. > # > # This stuff is extremely fragile, proceed with caution. > - set (luajit_cflags ${CMAKE_C_FLAGS}) > + string (TOUPPER ${CMAKE_BUILD_TYPE} upper_build_type) > + set (c_flags_init "CMAKE_C_FLAGS_${upper_build_type}") > + set (luajit_cflags ${${c_flags_init}}) Why do you set the new flags, instead of appending them to ${CMAKE_C_FLAGS}? If there is a particular reason, please drop a few words regarding it. Besides, I checked LuaJIT build flags: * prior to your patch | /usr/bin/cc -O2 -Wall -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG | -DNVALGRIND -DNVALGRIND=1 -DLUAJIT_SMART_STRINGS=1 -D_FILE_OFFSET_BITS=64 | -D_LARGEFILE_SOURCE -U_FORTIFY_SOURCE -fno-stack-protector -fexceptions | -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common | -fopenmp -msse2 -Wno-parentheses-equality -Wno-tautological-compare | -Wno-misleading-indent ation -Wno-varargs -Wno-implicit-fallthrough -c | -o lj_record.o lj_record.c * and with your changes | /usr/bin/cc -Wall -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG | -DNVALGRIND -DNVALGRIND=1 -DLUAJIT_SMART_STRINGS=1 -D_FILE_OFFSET_BITS=64 | -D_LARGEFILE_SOURCE -U_FORTIFY_SOURCE -fno-stack-protector -O2 -g -DNDEBUG | -ggdb -O2 -Wno-parentheses-equality -Wno-tautological-compare | -Wno-misleading-indentation -Wno-varargs -Wno-implicit-fallthrough -c | -o lj_record.o lj_record.c I have the same concerns Sasha mentioned in his review regarding -fexceptions and -funwind-tables flags. There are several issues in LuaJIT queue but it seems all them are related to the "host" program build (i.e. Tarantool), not LuaJIT itself. However, -fno-omit-frame-pointer is set for internal Tarantool purposes (despite Mike's comments in LuaJIT Makefile). So, I guess you can't freely drop them. > set (luajit_ldflags ${CMAKE_EXE_LINKER_FLAGS}) > separate_arguments(luajit_cflags) > separate_arguments(luajit_ldflags) > @@ -168,19 +170,12 @@ macro(luajit_build) > "-pagezero_size 10000 -image_base 100000000") > endif() > > - # We are consciously ommiting debug info in RelWithDebInfo mode Here I also agree with Sasha here: there was a reason to omit debuginfo for RelWithDebInfo builds, but you mentioned a word neither in commit message nor in a comment nearby. > + set (upper_build_type) > + set (c_flags_init) > + > if (${CMAKE_BUILD_TYPE} STREQUAL "Debug") > - set (luajit_ccopt -O0) > - if (CC_HAS_GGDB) > - set (luajit_ccdebug -g -ggdb) > - else () > - set (luajit_ccdebug -g) > - endif () > add_definitions(-DLUA_USE_APICHECK=1) > add_definitions(-DLUA_USE_ASSERT=1) > - else () > - set (luajit_ccopt -O2) > - set (luajit_ccdbebug "") > endif() > if (${CMAKE_SYSTEM_NAME} STREQUAL Darwin) > # Pass sysroot - prepended in front of system header/lib dirs, > -- > 2.20.1 > -- Best regards, IM From tsafin at tarantool.org Thu Aug 6 18:06:35 2020 From: tsafin at tarantool.org (Timur Safin) Date: Thu, 6 Aug 2020 18:06:35 +0300 Subject: [Tarantool-patches] [PATCH] Ensure all curl symbols are exported In-Reply-To: <20200806131955.3400088-1-yaroslav.dynnikov@tarantool.org> References: <20200806131955.3400088-1-yaroslav.dynnikov@tarantool.org> Message-ID: <07fa01d66c03$2da443d0$88eccb70$@tarantool.org> Much respect for the outstanding symbol loader test! LGTM Timur : From: Tarantool-patches On : Behalf Of Yaroslav Dynnikov : Subject: [Tarantool-patches] [PATCH] Ensure all curl symbols are exported : : In the recent update of libcurl (2.5.0-278-g807c7fa58) its layout has : changed: private function `Curl_version_init()` which used to fill-in : info structure was eliminated. As a result, no symbols for : `libcurl_la-version.o` remained used, so it wasn't included in tarantool : binary. And `curl_version` and `curl_version_info` symbols went missing. : : According to libcurl naming conventions all exported symbols are named : as `curl_*`. This patch lists them all explicitly in `exprots.h` and : adds the test. : : Close #5223 : : Issue: https://github.com/tarantool/tarantool/issues/5223 : Branch: https://github.com/tarantool/tarantool/tree/rosik/gh-5223-missing- : curl-symbols : : --- : src/exports.h | 81 ++++++++++ : test/box-tap/gh-5223-curl-exports.test.lua | 177 +++++++++++++++++++++ : 2 files changed, 258 insertions(+) : create mode 100755 test/box-tap/gh-5223-curl-exports.test.lua : : diff --git a/src/exports.h b/src/exports.h : index 7cf283e5b..7f0601f4f 100644 : --- a/src/exports.h : +++ b/src/exports.h : @@ -113,6 +113,87 @@ EXPORT(csv_feed) : EXPORT(csv_iterator_create) : EXPORT(csv_next) : EXPORT(csv_setopt) : +EXPORT(curl_easy_cleanup) : +EXPORT(curl_easy_duphandle) : +EXPORT(curl_easy_escape) : +EXPORT(curl_easy_getinfo) : +EXPORT(curl_easy_init) : +EXPORT(curl_easy_pause) : +EXPORT(curl_easy_perform) : +EXPORT(curl_easy_recv) : +EXPORT(curl_easy_reset) : +EXPORT(curl_easy_send) : +EXPORT(curl_easy_setopt) : +EXPORT(curl_easy_strerror) : +EXPORT(curl_easy_unescape) : +EXPORT(curl_easy_upkeep) : +EXPORT(curl_escape) : +EXPORT(curl_formadd) : +EXPORT(curl_formfree) : +EXPORT(curl_formget) : +EXPORT(curl_free) : +EXPORT(curl_getdate) : +EXPORT(curl_getenv) : +EXPORT(curl_global_cleanup) : +EXPORT(curl_global_init) : +EXPORT(curl_global_init_mem) : +EXPORT(curl_global_sslset) : +EXPORT(curl_maprintf) : +EXPORT(curl_mfprintf) : +EXPORT(curl_mime_addpart) : +EXPORT(curl_mime_data) : +EXPORT(curl_mime_data_cb) : +EXPORT(curl_mime_encoder) : +EXPORT(curl_mime_filedata) : +EXPORT(curl_mime_filename) : +EXPORT(curl_mime_free) : +EXPORT(curl_mime_headers) : +EXPORT(curl_mime_init) : +EXPORT(curl_mime_name) : +EXPORT(curl_mime_subparts) : +EXPORT(curl_mime_type) : +EXPORT(curl_mprintf) : +EXPORT(curl_msnprintf) : +EXPORT(curl_msprintf) : +EXPORT(curl_multi_add_handle) : +EXPORT(curl_multi_assign) : +EXPORT(curl_multi_cleanup) : +EXPORT(curl_multi_fdset) : +EXPORT(curl_multi_info_read) : +EXPORT(curl_multi_init) : +EXPORT(curl_multi_perform) : +EXPORT(curl_multi_poll) : +EXPORT(curl_multi_remove_handle) : +EXPORT(curl_multi_setopt) : +EXPORT(curl_multi_socket) : +EXPORT(curl_multi_socket_action) : +EXPORT(curl_multi_socket_all) : +EXPORT(curl_multi_strerror) : +EXPORT(curl_multi_timeout) : +EXPORT(curl_multi_wait) : +EXPORT(curl_mvaprintf) : +EXPORT(curl_mvfprintf) : +EXPORT(curl_mvprintf) : +EXPORT(curl_mvsnprintf) : +EXPORT(curl_mvsprintf) : +EXPORT(curl_pushheader_byname) : +EXPORT(curl_pushheader_bynum) : +EXPORT(curl_share_cleanup) : +EXPORT(curl_share_init) : +EXPORT(curl_share_setopt) : +EXPORT(curl_share_strerror) : +EXPORT(curl_slist_append) : +EXPORT(curl_slist_free_all) : +EXPORT(curl_strequal) : +EXPORT(curl_strnequal) : +EXPORT(curl_unescape) : +EXPORT(curl_url) : +EXPORT(curl_url_cleanup) : +EXPORT(curl_url_dup) : +EXPORT(curl_url_get) : +EXPORT(curl_url_set) : +EXPORT(curl_version) : +EXPORT(curl_version_info) : EXPORT(decimal_unpack) : EXPORT(error_ref) : EXPORT(error_set_prev) : diff --git a/test/box-tap/gh-5223-curl-exports.test.lua b/test/box-tap/gh- : 5223-curl-exports.test.lua : new file mode 100755 : index 000000000..300d60b07 : --- /dev/null : +++ b/test/box-tap/gh-5223-curl-exports.test.lua : @@ -0,0 +1,177 @@ : +#!/usr/bin/env tarantool : + : +local tap = require('tap') : +local ffi = require('ffi') : +ffi.cdef([[ : + void *dlsym(void *handle, const char *symbol); : + struct curl_version_info_data { : + int age; /* see description below */ : + const char *version; /* human readable string */ : + unsigned int version_num; /* numeric representation */ : + const char *host; /* human readable string */ : + int features; /* bitmask, see below */ : + char *ssl_version; /* human readable string */ : + long ssl_version_num; /* not used, always zero */ : + const char *libz_version; /* human readable string */ : + const char * const *protocols; /* protocols */ : + : + /* when 'age' is CURLVERSION_SECOND or higher, the members below : exist */ : + const char *ares; /* human readable string */ : + int ares_num; /* number */ : + : + /* when 'age' is CURLVERSION_THIRD or higher, the members below : exist */ : + const char *libidn; /* human readable string */ : + : + /* when 'age' is CURLVERSION_FOURTH or higher (>= 7.16.1), the : members : + below exist */ : + int iconv_ver_num; /* '_libiconv_version' if iconv support : enabled */ : + : + const char *libssh_version; /* human readable string */ : + : + /* when 'age' is CURLVERSION_FIFTH or higher (>= 7.57.0), the : members : + below exist */ : + unsigned int brotli_ver_num; /* Numeric Brotli version : + (MAJOR << 24) | (MINOR << 12) | : PATCH */ : + const char *brotli_version; /* human readable string. */ : + : + /* when 'age' is CURLVERSION_SIXTH or higher (>= 7.66.0), the : members : + below exist */ : + unsigned int nghttp2_ver_num; /* Numeric nghttp2 version : + (MAJOR << 16) | (MINOR << 8) | : PATCH */ : + const char *nghttp2_version; /* human readable string. */ : + : + const char *quic_version; /* human readable quic (+ HTTP/3) : library + : + version or NULL */ : + : + /* when 'age' is CURLVERSION_SEVENTH or higher (>= 7.70.0), the : members : + below exist */ : + const char *cainfo; /* the built-in default : CURLOPT_CAINFO, might : + be NULL */ : + const char *capath; /* the built-in default : CURLOPT_CAPATH, might : + be NULL */ : + }; : + : + struct curl_version_info_data *curl_version_info(int age); : +]]) : + : +local info = ffi.C.curl_version_info(7) : +local test = tap.test('curl-features') : +test:plan(3) : + : +if test:ok(info.ssl_version ~= nil, 'Curl built with SSL support') then : + test:diag('ssl_version: ' .. ffi.string(info.ssl_version)) : +end : +if test:ok(info.libz_version ~= nil, 'Curl built with LIBZ') then : + test:diag('libz_version: ' .. ffi.string(info.libz_version)) : +end : + : +local RTLD_DEFAULT : +-- See `man 3 dlsym`: : +-- RTLD_DEFAULT : +-- Find the first occurrence of the desired symbol using the default : +-- shared object search order. The search will include global symbols : +-- in the executable and its dependencies, as well as symbols in shared : +-- objects that were dynamically loaded with the RTLD_GLOBAL flag. : +if jit.os == "OSX" then : + RTLD_DEFAULT = ffi.cast("void *", -2LL) : +else : + RTLD_DEFAULT = ffi.cast("void *", 0LL) : +end : + : +-- The following list was obtained by parsing libcurl.a static library: : +-- nm libcurl.a | grep -oP 'T \K(curl_.+)$' | sort : +local curl_symbols = { : + 'curl_easy_cleanup', : + 'curl_easy_duphandle', : + 'curl_easy_escape', : + 'curl_easy_getinfo', : + 'curl_easy_init', : + 'curl_easy_pause', : + 'curl_easy_perform', : + 'curl_easy_recv', : + 'curl_easy_reset', : + 'curl_easy_send', : + 'curl_easy_setopt', : + 'curl_easy_strerror', : + 'curl_easy_unescape', : + 'curl_easy_upkeep', : + 'curl_escape', : + 'curl_formadd', : + 'curl_formfree', : + 'curl_formget', : + 'curl_free', : + 'curl_getdate', : + 'curl_getenv', : + 'curl_global_cleanup', : + 'curl_global_init', : + 'curl_global_init_mem', : + 'curl_global_sslset', : + 'curl_maprintf', : + 'curl_mfprintf', : + 'curl_mime_addpart', : + 'curl_mime_data', : + 'curl_mime_data_cb', : + 'curl_mime_encoder', : + 'curl_mime_filedata', : + 'curl_mime_filename', : + 'curl_mime_free', : + 'curl_mime_headers', : + 'curl_mime_init', : + 'curl_mime_name', : + 'curl_mime_subparts', : + 'curl_mime_type', : + 'curl_mprintf', : + 'curl_msnprintf', : + 'curl_msprintf', : + 'curl_multi_add_handle', : + 'curl_multi_assign', : + 'curl_multi_cleanup', : + 'curl_multi_fdset', : + 'curl_multi_info_read', : + 'curl_multi_init', : + 'curl_multi_perform', : + 'curl_multi_poll', : + 'curl_multi_remove_handle', : + 'curl_multi_setopt', : + 'curl_multi_socket', : + 'curl_multi_socket_action', : + 'curl_multi_socket_all', : + 'curl_multi_strerror', : + 'curl_multi_timeout', : + 'curl_multi_wait', : + 'curl_mvaprintf', : + 'curl_mvfprintf', : + 'curl_mvprintf', : + 'curl_mvsnprintf', : + 'curl_mvsprintf', : + 'curl_pushheader_byname', : + 'curl_pushheader_bynum', : + 'curl_share_cleanup', : + 'curl_share_init', : + 'curl_share_setopt', : + 'curl_share_strerror', : + 'curl_slist_append', : + 'curl_slist_free_all', : + 'curl_strequal', : + 'curl_strnequal', : + 'curl_unescape', : + 'curl_url', : + 'curl_url_cleanup', : + 'curl_url_dup', : + 'curl_url_get', : + 'curl_url_set', : + 'curl_version', : + 'curl_version_info', : +} : + : +test:test('curl_symbols', function(t) : + t:plan(#curl_symbols) : + for _, sym in ipairs(curl_symbols) do : + t:ok( : + ffi.C.dlsym(RTLD_DEFAULT, sym) ~= nil, : + ('Symbol %q found'):format(sym) : + ) : + end : +end) : + : +os.exit(test:check() and 0 or 1) : -- : 2.25.1 From olegrok at tarantool.org Thu Aug 6 19:00:43 2020 From: olegrok at tarantool.org (Oleg Babin) Date: Thu, 6 Aug 2020 19:00:43 +0300 Subject: [Tarantool-patches] [PATCH 1/2] tuple: fix multikey field JSON access crash In-Reply-To: References: Message-ID: <6a8870f3-ac79-7021-941d-11b90e687282@tarantool.org> Hi! Thanks for your patch. It's not a review but I have a question. On 05/08/2020 02:45, Vladislav Shpilevoy wrote: > When a tuple had format with multikey indexes in it, any attempt > to get a multikey indexed field by a JSON path from Lua led to a > crash. > > That was because of incorrect interpretation of offset slot value > in tuple's field map. > > Tuple field map is an array stored before the tuple's MessagePack > data. Each element is a 4 byte offset to an indexed value to be > able to get it for O(1) time without MessagePack decoding of all > the previous fields. > > At least it was so before multikeys. Now tuple field map is not > just an array. It is rather a 2-level array, somehow similar to > ext4 FS. Some elements of the root array are positive numbers > pointing at data. Some elements point at a second 'indirect' > array, so called 'extra', size of which is individual for each > tuple. These second arrays are used by multikey indexes to store > offsets to each multikey indexed value in a tuple. Do json path updates use offsets? Is such issue relevant for them? I tried to update poisoned tuple but seems it works fine. But maybe I've missed something. From v.shpilevoy at tarantool.org Thu Aug 6 22:57:41 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 21:57:41 +0200 Subject: [Tarantool-patches] [PATCH v2 vshard 0/2] storage: allow replica to boot before master In-Reply-To: References: Message-ID: <0ac0515a-5fab-66e7-c712-113f4efad26e@tarantool.org> Pushed to master. From v.shpilevoy at tarantool.org Thu Aug 6 23:04:10 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 22:04:10 +0200 Subject: [Tarantool-patches] [PATCH 1/2] tuple: fix multikey field JSON access crash In-Reply-To: <6a8870f3-ac79-7021-941d-11b90e687282@tarantool.org> References: <6a8870f3-ac79-7021-941d-11b90e687282@tarantool.org> Message-ID: <8892a38d-0608-314f-b32d-531db3bd6de3@tarantool.org> On 06.08.2020 18:00, Oleg Babin wrote: > Hi! Thanks for your patch. It's not a review but I have a question. > > On 05/08/2020 02:45, Vladislav Shpilevoy wrote: >> When a tuple had format with multikey indexes in it, any attempt >> to get a multikey indexed field by a JSON path from Lua led to a >> crash. >> >> That was because of incorrect interpretation of offset slot value >> in tuple's field map. >> >> Tuple field map is an array stored before the tuple's MessagePack >> data. Each element is a 4 byte offset to an indexed value to be >> able to get it for O(1) time without MessagePack decoding of all >> the previous fields. >> >> At least it was so before multikeys. Now tuple field map is not >> just an array. It is rather a 2-level array, somehow similar to >> ext4 FS. Some elements of the root array are positive numbers >> pointing at data. Some elements point at a second 'indirect' >> array, so called 'extra', size of which is individual for each >> tuple. These second arrays are used by multikey indexes to store >> offsets to each multikey indexed value in a tuple. > > > Do json path updates use offsets? Is such issue relevant for them? > > I tried to update poisoned tuple but seems it works fine. But maybe I've missed something. No, JSON updates always decode whole tuple, at least all fields <= max affected field. So offsets are not used. I was thinking about adding them, but so far there was no a request for it, nor benches how would it help exactly. From v.shpilevoy at tarantool.org Fri Aug 7 00:48:31 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Thu, 6 Aug 2020 23:48:31 +0200 Subject: [Tarantool-patches] [PATCH vshard 1/1] replicaset: check URI match when rebind connection Message-ID: <7f34d056c1f2b0f461740a4b04221881074106cd.1596750426.git.v.shpilevoy@tarantool.org> Reconfiguration in router and storage always creates new replica objects, and outdates old objects. Since most of the time reconfiguration does not change cluster topology and URIs, old connections are relocated to the new replicaset objects so as not to reconnect on each router.cfg() and storage.cfg(). Connections were moved from old objects to new ones by UUID match not taking into account possible URI change. If URI was changed, the new replica object took the old connection with old URI, and kept trying to connect to the old address. The patch makes connection relocation work only if old and new URI match. Closes #245 --- Branch: http://github.com/tarantool/vshard/tree/gerold103/gh-245-router-reconnect Issue: https://github.com/tarantool/vshard/issues/245 test/router/reconnect_to_master.result | 55 ++++++++++++++++++++++++ test/router/reconnect_to_master.test.lua | 26 +++++++++++ vshard/replicaset.lua | 2 +- 3 files changed, 82 insertions(+), 1 deletion(-) diff --git a/test/router/reconnect_to_master.result b/test/router/reconnect_to_master.result index d640f0b..aa33f9c 100644 --- a/test/router/reconnect_to_master.result +++ b/test/router/reconnect_to_master.result @@ -169,6 +169,61 @@ is_disconnected() --- - false ... +-- +-- gh-245: dynamic uri reconfiguration didn't work - even if URI was changed in +-- the config for any instance, it used old connection, because reconfiguration +-- compared connections by UUID instead of URI. +-- +util = require('util') +--- +... +-- Firstly, clean router from storage_1_a connection. +rs1_uuid = util.replicasets[1] +--- +... +rs1_cfg = cfg.sharding[rs1_uuid] +--- +... +cfg.sharding[rs1_uuid] = nil +--- +... +vshard.router.cfg(cfg) +--- +... +-- Now break the URI in the config. +old_uri = rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri +--- +... +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = 'https://bad_uri.com:123' +--- +... +-- Apply the bad config. +cfg.sharding[rs1_uuid] = rs1_cfg +--- +... +vshard.router.cfg(cfg) +--- +... +-- Should fail - master is not available because of the bad URI. +res, err = vshard.router.callrw(1, 'echo', {1}) +--- +... +res == nil and err ~= nil +--- +- true +... +-- Repair the config. +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = old_uri +--- +... +vshard.router.cfg(cfg) +--- +... +-- Should drop the old connection object and connect fine. +vshard.router.callrw(1, 'echo', {1}) +--- +- 1 +... _ = test_run:switch("default") --- ... diff --git a/test/router/reconnect_to_master.test.lua b/test/router/reconnect_to_master.test.lua index c315c9f..87270af 100644 --- a/test/router/reconnect_to_master.test.lua +++ b/test/router/reconnect_to_master.test.lua @@ -70,6 +70,32 @@ while is_disconnected() and i < max_iters do i = i + 1 fiber.sleep(0.1) end -- Master connection is active again. is_disconnected() +-- +-- gh-245: dynamic uri reconfiguration didn't work - even if URI was changed in +-- the config for any instance, it used old connection, because reconfiguration +-- compared connections by UUID instead of URI. +-- +util = require('util') +-- Firstly, clean router from storage_1_a connection. +rs1_uuid = util.replicasets[1] +rs1_cfg = cfg.sharding[rs1_uuid] +cfg.sharding[rs1_uuid] = nil +vshard.router.cfg(cfg) +-- Now break the URI in the config. +old_uri = rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = 'https://bad_uri.com:123' +-- Apply the bad config. +cfg.sharding[rs1_uuid] = rs1_cfg +vshard.router.cfg(cfg) +-- Should fail - master is not available because of the bad URI. +res, err = vshard.router.callrw(1, 'echo', {1}) +res == nil and err ~= nil +-- Repair the config. +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = old_uri +vshard.router.cfg(cfg) +-- Should drop the old connection object and connect fine. +vshard.router.callrw(1, 'echo', {1}) + _ = test_run:switch("default") _ = test_run:cmd('stop server router_1') _ = test_run:cmd('cleanup server router_1') diff --git a/vshard/replicaset.lua b/vshard/replicaset.lua index 0191936..b13d05e 100644 --- a/vshard/replicaset.lua +++ b/vshard/replicaset.lua @@ -456,7 +456,7 @@ local function rebind_replicasets(replicasets, old_replicasets) for replica_uuid, replica in pairs(replicaset.replicas) do local old_replica = old_replicaset and old_replicaset.replicas[replica_uuid] - if old_replica then + if old_replica and old_replica.uri == replica.uri then local conn = old_replica.conn replica.conn = conn replica.down_ts = old_replica.down_ts -- 2.21.1 (Apple Git-122.3) From yaroslav.dynnikov at tarantool.org Fri Aug 7 01:02:18 2020 From: yaroslav.dynnikov at tarantool.org (Yaroslav Dynnikov) Date: Fri, 7 Aug 2020 01:02:18 +0300 Subject: [Tarantool-patches] [PATCH vshard 1/1] replicaset: check URI match when rebind connection In-Reply-To: <7f34d056c1f2b0f461740a4b04221881074106cd.1596750426.git.v.shpilevoy@tarantool.org> References: <7f34d056c1f2b0f461740a4b04221881074106cd.1596750426.git.v.shpilevoy@tarantool.org> Message-ID: Hi! Thanks for the fix. LGTM. Best regards Yaroslav Dynnikov On Fri, 7 Aug 2020 at 00:48, Vladislav Shpilevoy wrote: > Reconfiguration in router and storage always creates new > replica objects, and outdates old objects. Since most of the > time reconfiguration does not change cluster topology and URIs, > old connections are relocated to the new replicaset objects so as > not to reconnect on each router.cfg() and storage.cfg(). > > Connections were moved from old objects to new ones by UUID match > not taking into account possible URI change. > > If URI was changed, the new replica object took the old connection > with old URI, and kept trying to connect to the old address. > > The patch makes connection relocation work only if old and new URI > match. > > Closes #245 > --- > Branch: > http://github.com/tarantool/vshard/tree/gerold103/gh-245-router-reconnect > Issue: https://github.com/tarantool/vshard/issues/245 > > test/router/reconnect_to_master.result | 55 ++++++++++++++++++++++++ > test/router/reconnect_to_master.test.lua | 26 +++++++++++ > vshard/replicaset.lua | 2 +- > 3 files changed, 82 insertions(+), 1 deletion(-) > > diff --git a/test/router/reconnect_to_master.result > b/test/router/reconnect_to_master.result > index d640f0b..aa33f9c 100644 > --- a/test/router/reconnect_to_master.result > +++ b/test/router/reconnect_to_master.result > @@ -169,6 +169,61 @@ is_disconnected() > --- > - false > ... > +-- > +-- gh-245: dynamic uri reconfiguration didn't work - even if URI was > changed in > +-- the config for any instance, it used old connection, because > reconfiguration > +-- compared connections by UUID instead of URI. > +-- > +util = require('util') > +--- > +... > +-- Firstly, clean router from storage_1_a connection. > +rs1_uuid = util.replicasets[1] > +--- > +... > +rs1_cfg = cfg.sharding[rs1_uuid] > +--- > +... > +cfg.sharding[rs1_uuid] = nil > +--- > +... > +vshard.router.cfg(cfg) > +--- > +... > +-- Now break the URI in the config. > +old_uri = rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri > +--- > +... > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = ' > https://bad_uri.com:123' > +--- > +... > +-- Apply the bad config. > +cfg.sharding[rs1_uuid] = rs1_cfg > +--- > +... > +vshard.router.cfg(cfg) > +--- > +... > +-- Should fail - master is not available because of the bad URI. > +res, err = vshard.router.callrw(1, 'echo', {1}) > +--- > +... > +res == nil and err ~= nil > +--- > +- true > +... > +-- Repair the config. > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = old_uri > +--- > +... > +vshard.router.cfg(cfg) > +--- > +... > +-- Should drop the old connection object and connect fine. > +vshard.router.callrw(1, 'echo', {1}) > +--- > +- 1 > +... > _ = test_run:switch("default") > --- > ... > diff --git a/test/router/reconnect_to_master.test.lua > b/test/router/reconnect_to_master.test.lua > index c315c9f..87270af 100644 > --- a/test/router/reconnect_to_master.test.lua > +++ b/test/router/reconnect_to_master.test.lua > @@ -70,6 +70,32 @@ while is_disconnected() and i < max_iters do i = i + 1 > fiber.sleep(0.1) end > -- Master connection is active again. > is_disconnected() > > +-- > +-- gh-245: dynamic uri reconfiguration didn't work - even if URI was > changed in > +-- the config for any instance, it used old connection, because > reconfiguration > +-- compared connections by UUID instead of URI. > +-- > +util = require('util') > +-- Firstly, clean router from storage_1_a connection. > +rs1_uuid = util.replicasets[1] > +rs1_cfg = cfg.sharding[rs1_uuid] > +cfg.sharding[rs1_uuid] = nil > +vshard.router.cfg(cfg) > +-- Now break the URI in the config. > +old_uri = rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = ' > https://bad_uri.com:123' > +-- Apply the bad config. > +cfg.sharding[rs1_uuid] = rs1_cfg > +vshard.router.cfg(cfg) > +-- Should fail - master is not available because of the bad URI. > +res, err = vshard.router.callrw(1, 'echo', {1}) > +res == nil and err ~= nil > +-- Repair the config. > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = old_uri > +vshard.router.cfg(cfg) > +-- Should drop the old connection object and connect fine. > +vshard.router.callrw(1, 'echo', {1}) > + > _ = test_run:switch("default") > _ = test_run:cmd('stop server router_1') > _ = test_run:cmd('cleanup server router_1') > diff --git a/vshard/replicaset.lua b/vshard/replicaset.lua > index 0191936..b13d05e 100644 > --- a/vshard/replicaset.lua > +++ b/vshard/replicaset.lua > @@ -456,7 +456,7 @@ local function rebind_replicasets(replicasets, > old_replicasets) > for replica_uuid, replica in pairs(replicaset.replicas) do > local old_replica = old_replicaset and > old_replicaset.replicas[replica_uuid] > - if old_replica then > + if old_replica and old_replica.uri == replica.uri then > local conn = old_replica.conn > replica.conn = conn > replica.down_ts = old_replica.down_ts > -- > 2.21.1 (Apple Git-122.3) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v.shpilevoy at tarantool.org Fri Aug 7 01:11:46 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Fri, 7 Aug 2020 00:11:46 +0200 Subject: [Tarantool-patches] [PATCH 1/1] router: update known bucket count when rs removed Message-ID: When replicaset was removed, router didn't update router.known_bucket_count value. It is used by discovery to decide when to turn on aggressive discovery. And by router.info() to provide alerts, change status. As a result, in case of replicaset removal, known_bucket_count was left bigger than it actually is, and it could be fixed only by restart. Discovery could become non-agressive too early, and router.info() could show weird alerts about bucke_count being configured differently on storages and router. Besides, router.info().bucket was showing not realistic numbers. --- Branch: http://github.com/tarantool/tarantool/tree/gerold103/router-known-buckets test/router/router2.result | 61 ++++++++++++++++++++++++++++++++++++ test/router/router2.test.lua | 24 ++++++++++++++ vshard/router/init.lua | 3 ++ 3 files changed, 88 insertions(+) diff --git a/test/router/router2.result b/test/router/router2.result index 40d6bff..492421b 100644 --- a/test/router/router2.result +++ b/test/router/router2.result @@ -250,6 +250,67 @@ vshard.router.static.discovery_fiber | - null | ... +-- +-- Known bucket count should be updated properly when replicaset +-- is removed from the config. +-- +vshard.router.info().bucket + | --- + | - unreachable: 0 + | available_ro: 0 + | unknown: 0 + | available_rw: 3000 + | ... +rs1_uuid = util.replicasets[1] + | --- + | ... +rs1 = cfg.sharding[rs1_uuid] + | --- + | ... +cfg.sharding[rs1_uuid] = nil + | --- + | ... +vshard.router.cfg(cfg) + | --- + | ... +vshard.router.info().bucket + | --- + | - unreachable: 0 + | available_ro: 0 + | unknown: 1500 + | available_rw: 1500 + | ... +cfg.sharding[rs1_uuid] = rs1 + | --- + | ... +vshard.router.cfg(cfg) + | --- + | ... +vshard.router.discovery_set('on') + | --- + | ... +function wait_all_rw() \ + local total = vshard.router.bucket_count() \ + local res = vshard.router.info().bucket.available_rw == total \ + if not res then \ + vshard.router.discovery_wakeup() \ + end \ + return res \ +end + | --- + | ... +test_run:wait_cond(wait_all_rw) + | --- + | - true + | ... +vshard.router.info().bucket + | --- + | - unreachable: 0 + | available_ro: 0 + | unknown: 0 + | available_rw: 3000 + | ... + _ = test_run:switch("default") | --- | ... diff --git a/test/router/router2.test.lua b/test/router/router2.test.lua index afdf4e1..72c99e6 100644 --- a/test/router/router2.test.lua +++ b/test/router/router2.test.lua @@ -95,6 +95,30 @@ vshard.router.static.discovery_fiber vshard.router.discovery_set('once') vshard.router.static.discovery_fiber +-- +-- Known bucket count should be updated properly when replicaset +-- is removed from the config. +-- +vshard.router.info().bucket +rs1_uuid = util.replicasets[1] +rs1 = cfg.sharding[rs1_uuid] +cfg.sharding[rs1_uuid] = nil +vshard.router.cfg(cfg) +vshard.router.info().bucket +cfg.sharding[rs1_uuid] = rs1 +vshard.router.cfg(cfg) +vshard.router.discovery_set('on') +function wait_all_rw() \ + local total = vshard.router.bucket_count() \ + local res = vshard.router.info().bucket.available_rw == total \ + if not res then \ + vshard.router.discovery_wakeup() \ + end \ + return res \ +end +test_run:wait_cond(wait_all_rw) +vshard.router.info().bucket + _ = test_run:switch("default") _ = test_run:cmd("stop server router_1") _ = test_run:cmd("cleanup server router_1") diff --git a/vshard/router/init.lua b/vshard/router/init.lua index d8c9246..0d94210 100644 --- a/vshard/router/init.lua +++ b/vshard/router/init.lua @@ -908,14 +908,17 @@ local function router_cfg(router, cfg, is_reload) router.failover_ping_timeout = vshard_cfg.failover_ping_timeout router.sync_timeout = vshard_cfg.sync_timeout local old_route_map = router.route_map + local known_bucket_count = 0 router.route_map = table_new(router.total_bucket_count, 0) for bucket, rs in pairs(old_route_map) do local new_rs = router.replicasets[rs.uuid] if new_rs then router.route_map[bucket] = new_rs new_rs.bucket_count = new_rs.bucket_count + 1 + known_bucket_count = known_bucket_count + 1 end end + router.known_bucket_count = known_bucket_count if router.failover_fiber == nil then router.failover_fiber = util.reloadable_fiber_create( 'vshard.failover.' .. router.name, M, 'failover_f', router) -- 2.21.1 (Apple Git-122.3) From v.shpilevoy at tarantool.org Fri Aug 7 01:14:13 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Fri, 7 Aug 2020 00:14:13 +0200 Subject: [Tarantool-patches] [PATCH 1/1] router: update known bucket count when rs removed In-Reply-To: References: Message-ID: <4f828799-394e-c6d7-c6a5-700f87a55896@tarantool.org> Forgot to add 'vshard' after 'PATCH' in the title. Also the real branch is http://github.com/tarantool/vshard/tree/gerold103/router-known-buckets (I generate patch emails with a script, and forgot to pass the option with repository name.) From olegrok at tarantool.org Fri Aug 7 11:07:16 2020 From: olegrok at tarantool.org (Oleg Babin) Date: Fri, 7 Aug 2020 11:07:16 +0300 Subject: [Tarantool-patches] [PATCH 1/1] router: update known bucket count when rs removed In-Reply-To: References: Message-ID: Thanks for your patch! LGTM. On 07/08/2020 01:11, Vladislav Shpilevoy wrote: > When replicaset was removed, router didn't update > router.known_bucket_count value. It is used by discovery to decide > when to turn on aggressive discovery. And by router.info() to > provide alerts, change status. > > As a result, in case of replicaset removal, known_bucket_count > was left bigger than it actually is, and it could be fixed only > by restart. > > Discovery could become non-agressive too early, and router.info() > could show weird alerts about bucke_count being configured > differently on storages and router. Besides, router.info().bucket > was showing not realistic numbers. > --- > Branch:http://github.com/tarantool/tarantool/tree/gerold103/router-known-buckets > > test/router/router2.result | 61 ++++++++++++++++++++++++++++++++++++ > test/router/router2.test.lua | 24 ++++++++++++++ > vshard/router/init.lua | 3 ++ > 3 files changed, 88 insertions(+) > > diff --git a/test/router/router2.result b/test/router/router2.result > index 40d6bff..492421b 100644 > --- a/test/router/router2.result > +++ b/test/router/router2.result > @@ -250,6 +250,67 @@ vshard.router.static.discovery_fiber > | - null > | ... > > +-- > +-- Known bucket count should be updated properly when replicaset > +-- is removed from the config. > +-- > +vshard.router.info().bucket > + | --- > + | - unreachable: 0 > + | available_ro: 0 > + | unknown: 0 > + | available_rw: 3000 > + | ... > +rs1_uuid = util.replicasets[1] > + | --- > + | ... > +rs1 = cfg.sharding[rs1_uuid] > + | --- > + | ... > +cfg.sharding[rs1_uuid] = nil > + | --- > + | ... > +vshard.router.cfg(cfg) > + | --- > + | ... > +vshard.router.info().bucket > + | --- > + | - unreachable: 0 > + | available_ro: 0 > + | unknown: 1500 > + | available_rw: 1500 > + | ... > +cfg.sharding[rs1_uuid] = rs1 > + | --- > + | ... > +vshard.router.cfg(cfg) > + | --- > + | ... > +vshard.router.discovery_set('on') > + | --- > + | ... > +function wait_all_rw() \ > + local total = vshard.router.bucket_count() \ > + local res = vshard.router.info().bucket.available_rw == total \ > + if not res then \ > + vshard.router.discovery_wakeup() \ > + end \ > + return res \ > +end > + | --- > + | ... > +test_run:wait_cond(wait_all_rw) > + | --- > + | - true > + | ... > +vshard.router.info().bucket > + | --- > + | - unreachable: 0 > + | available_ro: 0 > + | unknown: 0 > + | available_rw: 3000 > + | ... > + > _ = test_run:switch("default") > | --- > | ... > diff --git a/test/router/router2.test.lua b/test/router/router2.test.lua > index afdf4e1..72c99e6 100644 > --- a/test/router/router2.test.lua > +++ b/test/router/router2.test.lua > @@ -95,6 +95,30 @@ vshard.router.static.discovery_fiber > vshard.router.discovery_set('once') > vshard.router.static.discovery_fiber > > +-- > +-- Known bucket count should be updated properly when replicaset > +-- is removed from the config. > +-- > +vshard.router.info().bucket > +rs1_uuid = util.replicasets[1] > +rs1 = cfg.sharding[rs1_uuid] > +cfg.sharding[rs1_uuid] = nil > +vshard.router.cfg(cfg) > +vshard.router.info().bucket > +cfg.sharding[rs1_uuid] = rs1 > +vshard.router.cfg(cfg) > +vshard.router.discovery_set('on') > +function wait_all_rw() \ > + local total = vshard.router.bucket_count() \ > + local res = vshard.router.info().bucket.available_rw == total \ > + if not res then \ > + vshard.router.discovery_wakeup() \ > + end \ > + return res \ > +end > +test_run:wait_cond(wait_all_rw) > +vshard.router.info().bucket > + > _ = test_run:switch("default") > _ = test_run:cmd("stop server router_1") > _ = test_run:cmd("cleanup server router_1") > diff --git a/vshard/router/init.lua b/vshard/router/init.lua > index d8c9246..0d94210 100644 > --- a/vshard/router/init.lua > +++ b/vshard/router/init.lua > @@ -908,14 +908,17 @@ local function router_cfg(router, cfg, is_reload) > router.failover_ping_timeout = vshard_cfg.failover_ping_timeout > router.sync_timeout = vshard_cfg.sync_timeout > local old_route_map = router.route_map > + local known_bucket_count = 0 > router.route_map = table_new(router.total_bucket_count, 0) > for bucket, rs in pairs(old_route_map) do > local new_rs = router.replicasets[rs.uuid] > if new_rs then > router.route_map[bucket] = new_rs > new_rs.bucket_count = new_rs.bucket_count + 1 > + known_bucket_count = known_bucket_count + 1 > end > end > + router.known_bucket_count = known_bucket_count > if router.failover_fiber == nil then > router.failover_fiber = util.reloadable_fiber_create( > 'vshard.failover.' .. router.name, M, 'failover_f', router) From olegrok at tarantool.org Fri Aug 7 11:07:22 2020 From: olegrok at tarantool.org (Oleg Babin) Date: Fri, 7 Aug 2020 11:07:22 +0300 Subject: [Tarantool-patches] [PATCH vshard 1/1] replicaset: check URI match when rebind connection In-Reply-To: <7f34d056c1f2b0f461740a4b04221881074106cd.1596750426.git.v.shpilevoy@tarantool.org> References: <7f34d056c1f2b0f461740a4b04221881074106cd.1596750426.git.v.shpilevoy@tarantool.org> Message-ID: <9b0b6fad-f296-aec9-881a-1efe13e89f72@tarantool.org> Hi! Thanks for your patch. LGTM. On 07/08/2020 00:48, Vladislav Shpilevoy wrote: > Reconfiguration in router and storage always creates new > replica objects, and outdates old objects. Since most of the > time reconfiguration does not change cluster topology and URIs, > old connections are relocated to the new replicaset objects so as > not to reconnect on each router.cfg() and storage.cfg(). > > Connections were moved from old objects to new ones by UUID match > not taking into account possible URI change. > > If URI was changed, the new replica object took the old connection > with old URI, and kept trying to connect to the old address. > > The patch makes connection relocation work only if old and new URI > match. > > Closes #245 > --- > Branch:http://github.com/tarantool/vshard/tree/gerold103/gh-245-router-reconnect > Issue:https://github.com/tarantool/vshard/issues/245 > > test/router/reconnect_to_master.result | 55 ++++++++++++++++++++++++ > test/router/reconnect_to_master.test.lua | 26 +++++++++++ > vshard/replicaset.lua | 2 +- > 3 files changed, 82 insertions(+), 1 deletion(-) > > diff --git a/test/router/reconnect_to_master.result b/test/router/reconnect_to_master.result > index d640f0b..aa33f9c 100644 > --- a/test/router/reconnect_to_master.result > +++ b/test/router/reconnect_to_master.result > @@ -169,6 +169,61 @@ is_disconnected() > --- > - false > ... > +-- > +-- gh-245: dynamic uri reconfiguration didn't work - even if URI was changed in > +-- the config for any instance, it used old connection, because reconfiguration > +-- compared connections by UUID instead of URI. > +-- > +util = require('util') > +--- > +... > +-- Firstly, clean router from storage_1_a connection. > +rs1_uuid = util.replicasets[1] > +--- > +... > +rs1_cfg = cfg.sharding[rs1_uuid] > +--- > +... > +cfg.sharding[rs1_uuid] = nil > +--- > +... > +vshard.router.cfg(cfg) > +--- > +... > +-- Now break the URI in the config. > +old_uri = rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri > +--- > +... > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = 'https://bad_uri.com:123' > +--- > +... > +-- Apply the bad config. > +cfg.sharding[rs1_uuid] = rs1_cfg > +--- > +... > +vshard.router.cfg(cfg) > +--- > +... > +-- Should fail - master is not available because of the bad URI. > +res, err = vshard.router.callrw(1, 'echo', {1}) > +--- > +... > +res == nil and err ~= nil > +--- > +- true > +... > +-- Repair the config. > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = old_uri > +--- > +... > +vshard.router.cfg(cfg) > +--- > +... > +-- Should drop the old connection object and connect fine. > +vshard.router.callrw(1, 'echo', {1}) > +--- > +- 1 > +... > _ = test_run:switch("default") > --- > ... > diff --git a/test/router/reconnect_to_master.test.lua b/test/router/reconnect_to_master.test.lua > index c315c9f..87270af 100644 > --- a/test/router/reconnect_to_master.test.lua > +++ b/test/router/reconnect_to_master.test.lua > @@ -70,6 +70,32 @@ while is_disconnected() and i < max_iters do i = i + 1 fiber.sleep(0.1) end > -- Master connection is active again. > is_disconnected() > > +-- > +-- gh-245: dynamic uri reconfiguration didn't work - even if URI was changed in > +-- the config for any instance, it used old connection, because reconfiguration > +-- compared connections by UUID instead of URI. > +-- > +util = require('util') > +-- Firstly, clean router from storage_1_a connection. > +rs1_uuid = util.replicasets[1] > +rs1_cfg = cfg.sharding[rs1_uuid] > +cfg.sharding[rs1_uuid] = nil > +vshard.router.cfg(cfg) > +-- Now break the URI in the config. > +old_uri = rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = 'https://bad_uri.com:123' > +-- Apply the bad config. > +cfg.sharding[rs1_uuid] = rs1_cfg > +vshard.router.cfg(cfg) > +-- Should fail - master is not available because of the bad URI. > +res, err = vshard.router.callrw(1, 'echo', {1}) > +res == nil and err ~= nil > +-- Repair the config. > +rs1_cfg.replicas[util.name_to_uuid.storage_1_a].uri = old_uri > +vshard.router.cfg(cfg) > +-- Should drop the old connection object and connect fine. > +vshard.router.callrw(1, 'echo', {1}) > + > _ = test_run:switch("default") > _ = test_run:cmd('stop server router_1') > _ = test_run:cmd('cleanup server router_1') > diff --git a/vshard/replicaset.lua b/vshard/replicaset.lua > index 0191936..b13d05e 100644 > --- a/vshard/replicaset.lua > +++ b/vshard/replicaset.lua > @@ -456,7 +456,7 @@ local function rebind_replicasets(replicasets, old_replicasets) > for replica_uuid, replica in pairs(replicaset.replicas) do > local old_replica = old_replicaset and > old_replicaset.replicas[replica_uuid] > - if old_replica then > + if old_replica and old_replica.uri == replica.uri then > local conn = old_replica.conn > replica.conn = conn > replica.down_ts = old_replica.down_ts