From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp20.mail.ru (smtp20.mail.ru [94.100.179.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C1EC1469719 for ; Thu, 15 Oct 2020 16:18:17 +0300 (MSK) Date: Thu, 15 Oct 2020 16:18:37 +0300 From: Alexander Turenko Message-ID: <20201015131837.dincdgu4p6pws6xz@tkn_work_nb> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Subject: Re: [Tarantool-patches] [PATCH v3 15/16] module api: add box_key_def_validate_key() List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladislav Shpilevoy Cc: tarantool-patches@dev.tarantool.org On Thu, Oct 15, 2020 at 01:41:50AM +0200, Vladislav Shpilevoy wrote: > Thanks for the patch! > > See 2 comments below. > > On 13.10.2020 01:23, Alexander Turenko wrote: > > The function allows to verify a key against a key definition. It accepts > > a partial key. > > > > Part of #5273 > > --- > > src/box/key_def.c | 13 +++++ > > src/box/key_def.h | 21 ++++++++ > > src/exports.h | 1 + > > test/app-tap/module_api.c | 88 ++++++++++++++++++++++++++++++++ > > test/app-tap/module_api.test.lua | 2 +- > > 5 files changed, 124 insertions(+), 1 deletion(-) > > > > diff --git a/src/box/key_def.c b/src/box/key_def.c > > index da1c23135..55ac79362 100644 > > --- a/src/box/key_def.c > > +++ b/src/box/key_def.c > > @@ -627,6 +627,19 @@ box_key_def_extract_key(box_key_def_t *key_def, box_tuple_t *tuple, > > return tuple_extract_key(tuple, key_def, multikey_idx, key_size_ptr); > > } > > > > +int > > +box_key_def_validate_key(const box_key_def_t *key_def, const char *key) > > +{ > > + uint32_t part_count = mp_decode_array(&key); > > + if (part_count > key_def->part_count) { > > + diag_set(ClientError, ER_KEY_PART_COUNT, key_def->part_count, > > + part_count); > > + return -1; > > + } > > + const char *key_end; > > 1. Don't you want to make it an out parameter? We won't be able to add it > later, in case it will be ever needed. It seems to be quite specific case: when we validate an array of keys. However, okay, why not? I'll do it consistent with _extract_key(): . I'll also eliminate all those problems with const, when a caller-side is infected by const and must be . It'll be a bit intruisive on 1.10: several usages of key_validate_parts() must be updated like it was done on 2.* somewhere around key_list introduction. Added on the branches (for box_key_def_validate_full_key() as well). The change looks so on master: diff --git a/src/box/key_def.c b/src/box/key_def.c index 55ac79362..ca8a00dc5 100644 --- a/src/box/key_def.c +++ b/src/box/key_def.c @@ -628,16 +628,20 @@ box_key_def_extract_key(box_key_def_t *key_def, box_tuple_t *tuple, } int -box_key_def_validate_key(const box_key_def_t *key_def, const char *key) +box_key_def_validate_key(const box_key_def_t *key_def, const char *key, + uint32_t *key_size_ptr) { - uint32_t part_count = mp_decode_array(&key); + const char *pos = key; + uint32_t part_count = mp_decode_array(&pos); if (part_count > key_def->part_count) { diag_set(ClientError, ER_KEY_PART_COUNT, key_def->part_count, part_count); return -1; } - const char *key_end; - return key_validate_parts(key_def, key, part_count, true, &key_end); + int rc = key_validate_parts(key_def, pos, part_count, true, &pos); + if (rc == 0 && key_size_ptr != NULL) + *key_size_ptr = pos - key; + return rc; } /* }}} Module API functions */ diff --git a/src/box/key_def.h b/src/box/key_def.h index 8d942ee8f..9a0339757 100644 --- a/src/box/key_def.h +++ b/src/box/key_def.h @@ -557,14 +557,19 @@ box_key_def_extract_key(box_key_def_t *key_def, box_tuple_t *tuple, * * Note: nil is accepted for nullable fields, but only for them. * - * @param key_def Key definition. - * @param key MessagePack'ed data for matching. + * @param key_def Key definition. + * @param key MessagePack'ed data for matching. + * @param key_size_ptr Here will be size of the validated key. * * @retval 0 The key is valid. * @retval -1 The key is invalid. + * + * In case of an error set a diag and return -1. + * @sa (). */ API_EXPORT int -box_key_def_validate_key(const box_key_def_t *key_def, const char *key); +box_key_def_validate_key(const box_key_def_t *key_def, const char *key, + uint32_t *key_size_ptr); /** \endcond public */ diff --git a/test/app-tap/module_api.c b/test/app-tap/module_api.c index 778e7c3c0..f13b4be38 100644 --- a/test/app-tap/module_api.c +++ b/test/app-tap/module_api.c @@ -1746,14 +1746,17 @@ test_key_def_validate_key(struct lua_State *L) * | 5 | [1, 2, 3] | invalid | * | 6 | [1, -1] | invalid | */ - const char *keys[] = { - /* [0] = */ "\x92\x01\x01", - /* [1] = */ "\x92\x01\xc0", - /* [2] = */ "\x91\x01", - /* [3] = */ "\x90", - /* [4] = */ "\x91\xc0", - /* [5] = */ "\x93\x01\x02\x03", - /* [6] = */ "\x92\x01\xff", + struct { + const char *data; + uint32_t size; + } keys[] = { + /* [0] = */ {"\x92\x01\x01", 3}, + /* [1] = */ {"\x92\x01\xc0", 3}, + /* [2] = */ {"\x91\x01", 2}, + /* [3] = */ {"\x90", 1}, + /* [4] = */ {"\x91\xc0", 2}, + /* [5] = */ {"\x93\x01\x02\x03", 4}, + /* [6] = */ {"\x92\x01\xff", 3}, }; int expected_results[] = { /* [0] = */ 0, @@ -1775,10 +1778,23 @@ test_key_def_validate_key(struct lua_State *L) }; for (size_t i = 0; i < lengthof(keys); ++i) { - int rc = box_key_def_validate_key(key_def, keys[i]); + uint32_t key_size = 0; + const char *key = keys[i].data; + int rc = box_key_def_validate_key(key_def, key, &key_size); assert(rc == expected_results[i]); - if (expected_error_codes[i] != box_error_code_MAX) { + if (expected_error_codes[i] == box_error_code_MAX) { + /* Verify key_size. */ + assert(key_size != 0); + assert(key_size == keys[i].size); + + /* + * Verify that no NULL pointer dereference + * occurs when NULL is passed as + * key_size_ptr. + */ + box_key_def_validate_key(key_def, key, NULL); + } else { assert(rc != 0); box_error_t *e = box_error_last(); assert(box_error_code(e) == expected_error_codes[i]); > > > + return key_validate_parts(key_def, key, part_count, true, &key_end); > > +} > > + > > /* }}} Module API functions */ > > > > diff --git a/src/box/key_def.h b/src/box/key_def.h > > index 1b27836a8..440f2fb13 100644 > > --- a/src/box/key_def.h > > +++ b/src/box/key_def.h > > @@ -545,6 +545,27 @@ API_EXPORT char * > > box_key_def_extract_key(box_key_def_t *key_def, box_tuple_t *tuple, > > int multikey_idx, uint32_t *key_size_ptr); > > > > +/** > > + * Check a key against given key definition. > > + * > > + * Verifies key parts against given key_def's field types with > > + * respect to nullability. > > + * > > + * A partial key (with less part than defined in @a key_def) is > > + * verified by given key parts, the omitted tail is not verified > > + * anyhow. > > + * > > + * Note: nil is accepted for nullable fields, but only for them. > > + * > > + * @param key_def Key definition. > > + * @param key MessagePack'ed data for matching. > > + * > > + * @retval 0 The key is valid. > > + * @retval -1 The key is invalid. > > 2. Maybe it is worth saying in all the new public functions, that > if they return -1, it means diag is set, and is available via > box_error_message() and shit. Sure. Added phrases as follows on the branches: | In case of an error set a diag and return -1. | @sa (). | In case of an error set a diag and return NULL. | @sa (). | In case of an invalid tuple set a diag and return -1. | @sa (). | In case of an invalid key set a diag and return -1. | @sa (). The list of functions, where I did it: - box_key_def_new_v2 - box_key_def_dump_parts - box_key_def_validate_tuple - box_key_def_merge - box_key_def_extract_key - box_key_def_validate_key - box_key_def_validate_full_key The rest of added functions already mention the diagnostics area somehow. Aside of this I observed that box_region_alloc() and box_region_aligned_alloc() must also set a diag, because they're box api functions. Fixed this on the branches. diff --git a/src/lib/core/fiber.c b/src/lib/core/fiber.c index 33ec47c66..fab7740b2 100644 --- a/src/lib/core/fiber.c +++ b/src/lib/core/fiber.c @@ -1377,13 +1377,19 @@ box_region_used(void) void * box_region_alloc(size_t size) { - return region_alloc(&fiber()->gc, size); + void *res = region_alloc(&fiber()->gc, size); + if (res == NULL) + diag_set(OutOfMemory, size, "region_alloc", "data"); + return res; } void * box_region_aligned_alloc(size_t size, size_t alignment) { - return region_aligned_alloc(&fiber()->gc, size, alignment); + void *res = region_aligned_alloc(&fiber()->gc, size, alignment); + if (res == NULL) + diag_set(OutOfMemory, size, "region_alloc", "aligned data"); + return res; } void diff --git a/src/lib/core/fiber.h b/src/lib/core/fiber.h index ca3028b20..539e5c8e7 100644 --- a/src/lib/core/fiber.h +++ b/src/lib/core/fiber.h @@ -444,6 +444,9 @@ box_region_used(void); * or array of values of a type with alignment requirements. A * violation of alignment requirements leads to undefined * behaviour. + * + * In case of a memory error set a diag and return NULL. + * @sa (). */ API_EXPORT void * box_region_alloc(size_t size); @@ -452,6 +455,9 @@ box_region_alloc(size_t size); * Allocate size bytes from the box region with given alignment. * * Alignment must be a power of 2. + * + * In case of a memory error set a diag and return NULL. + * @sa (). */ API_EXPORT void * box_region_aligned_alloc(size_t size, size_t alignment);